Conformance Checking and Pushdown Reactive Systems

Due to their asynchronous interactions, testing reactive systems is a laborious activity present in any software development project. In this setting, the ﬁnite memory formalism of Labeled Transition Systems has been used to generate test suites that can be applied to check ioco conformance of implementations to a given speciﬁcation. In this work we turn to a more complex scenario where a stronger formalism is considered, the Visibly Pushdown Labeled Transition System (VPTS), which allows access to a potentially in-ﬁnite pushdown memory . We study an extension of the ioco conformance relation to VPTS models and develop polynomial time algorithms to verify conformance for VPTS models in a white-box testing scenario.


Introduction
Reactive systems have become increasingly common among computer systems, whether used in simple technological solutions or in critical industrial applications. We see everywhere real-world systems being governed by reactive behaviors where the systems interact with an external environment by receiving input stimuli and producing outputs in response. Usually, the development of such systems requires precise and automatic support, especially in the testing activity, because high costs in terms of resources and maintenance time can be incurred when the test step is inappropriately performed.
Model-based testing is an important approach that has been employed to test reactive systems because it offers guarantees to the correctness of important properties, such as completeness of test suites, which can be formally proven [1,2]. These aspects have been studied using appropriate models that capture the behavior of reactive systems, where the exchange of input and output stimuli can occur asynchronously. Prominent among such formalisms are Input/Output Labeled Transition Systems (IOLTSs) [3].
The process of testing is usually designed to verify whether an implementation is in compliance to a given specification [3]. This conformance checking process depends on the specification formalism, on the kind of fault model used, and on a conformance relation that is to be verified [3,4]. For IOLTS models, Input/Output Conformance (ioco) is a well-studied relation [3,5]. Recently, a more general approach [6], based on regular languages, has been proposed to check ioco conformance for IOLTS models.
In this work we mainly focus on pushdown reactive systems, more specifically, on a conformance checking framework for systems of this nature. We are not aware of any other approach that gives an efficient algorithm for checking conformance when treating models equipped with a pushdown memory. We study aspects of conformance testing and test suite generation for Input/Output Visibly Pushdown Labeled Transition (IOVPTS) models, inspired by the Visibly Pushdown Automata [7] formalism. Using an auxiliary pushdown stack, an IOVPTS can capture the behavior of much more complex reactive systems when compared to the simpler IOLTS models. Such is the case of an automatic vending machine that must keep track of the amount inserted, subtract the value of the purchase, and return the appropriate change. A proper model for such systems must use a formalism that allows for the manipulation of a potentially infinite memory. An illustration is discussed in Subsection 5.2 with the typical example of a drink dispensing machine, whose model requires a potentially infinite memory to formally describe its behavior as a reactive system. The same situation may also arise with other typical computational systems that are implemented via recursive programs. See Section 2 for comments on some related works that treat some aspects of recursive programs in the context of conformance testing.
Therefore we propose an extension of the ioco relation to IOVPTS models, in the sense that it prevents any observable implementation behavior that was not already present in the given specification. Our approach is based on a white-box testing scenario, where the structure of the implementation under test (IUT) is previously known by the tester. Specifications are assumed deterministic, but IUTSs can be even non-deterministic, and no further restrictions are required to constrain either on the specification or in the IUT models. The main novelty in this setting is to formally address the problem of developing algorithms of an acceptable complexity while still formally dealing with a potentially infinite pushdown memory.
Our algorithms always reach conclusive verdicts, either for conformance or for non-conformance, in polynomial time. When non-conformance is detected, a trace that witnesses the non-conformance is also generated. By contrast, in black-box scenarios, some restrictions must be assumed over the models and also limitations over the results. We prove the correctness of our algorithms and show that they run in worst case polynomial time in the size of both the specification and implementation.
We organize this paper as follows. Section 2 comments on some related works. In Section 3 we establish notations, define IOVPTSs and give preliminary results. Section 4 defines an ioco-like conformance relation for IOVPTS models and shows how to construct complete test suites. Section 5 develops an algorithm to test IUTs, proves its correctness, establishes a polynomial time bound for it, and illustrates these ideas on practical scenarios. Section 6 offers concluding remarks.

Related Works
Testing reactive systems has been largely studied in the literature, but several challenges still remain, mainly due to system and environment intrinsic complexities. Many researches of model-based testing usually deal with classical reactive systems, that is, they can be modeled by formalisms without memory, such as Labeled Transition Systems (LTSs) and their extension the Input/Output Labeled Transition Systems (IOLTSs).
In this work we study implementation testing in a scenario involving VPTSs, and their IOVPTS extensions, which are more complex and powerful models since they can access an unlimited pushdown memory. We develop a polynomial checking method that can be used to certify conformance between a given IOVPTS specification and a tentative IOVPTS implementation in a white-box testing scenario, where the structure of the implementation is known. Specifications models are assumed to be deterministic, but in the implementation side we can have non-deterministic models.
We now comment on some works more closely related to our approach. Constant et al. [8] propose a test case generation method for checking conformance of reactive systems in a black-box setting. Their method is based on algorithms that transform a set of recursive interprocedural specifications, with the semantics of the whole system being modeled as Push-Down Systems (PDSs). We note that the semantics of their formalism captures the so called Dyck languages, where each corresponding pair of symbols -the procedure calls and returns -appears properly balanced. These languages are a proper subset of the Visibly Pushdown Languages (VPLs), whose full extension is considered in this work. Their approach is confined to deterministic models and does not deal explicitly with arbitrary internal symbols as we do. Also quiescence is not treated, an important issue in a black-box scenario. Since they make use of test purposes, modeled by simple LTSs, their algorithms might lead to inconclusive verdicts when a state is reached from which it is impossible to reach an accept state in the test purpose model. In our scenario, conclusive verdicts are always emitted.
In more recent works, Dyck languages are also addressed in terms of the Dyck reachability problem which, in turn, is reduced to a reachability analysis when searching for a witness on checking conformance. Li et al. [9] propose a Dyck reachability analysis in time complexity O(n 7 ) when considering two Dyck languages. Kjelstrøm and Pavlogiannis [10] have treated the interleaved Dyck reachability problem and their different variants. They have found an efficient algorithm for the interleaved Dyck reachability problem for two Dyck languages with a time complexity bounded by O(n 3 α(n)), where α(n) is the inverse Ackermann function [11]. However, as we noticed the class of Dyck languages is a proper subset of the VPLs, so confining their results in a more restrict group of applications when compared to our proposal.
Chédor et al. [12] propose a test case generation method where specifications are modeled as the so called Recursive Tile Systems (RTS), whose semantics is captured by the traces of an IOLTS with infinite states. They proceed by applying successive transformations to the original RTS that models the specification, and the testing scenario treats implementations as black-boxes. At each step, properties of the resulting RTSs can then be established by examining the traces of the corresponding infinite IOLTSs. As the authors acknowledge, their approach can lead to test cases with an exponential state-space complexity, even if the original specification models are deterministic. In our method we generate, in polynomial time, test cases which provides complete fault coverage in a white-box scenario, and when specifications are deterministic. Since they also make use of test purposes as an aid to the certification process, their testing algorithms can lead to inconclusive verdicts when the testing process reaches where test purpose state marked Accept can no longer be reached. Other approaches were designed using reachability analyses and model checking algorithms as proposed by Esparza et al. [13] and Finkel et al. [14]. In part, our approach is also inspired by some of these ideas, such as when checking for balanced runs, in Section 5. Some other researches also focus on the synchronization problem of Input-driven Pushdown Automata [15,16,17], where witness configurations must be found over an infinite state transition system. All these approaches lead to polynomial time algorithms as mentioned in the previous related works. However, we are not aware of any other approach that deal with the conformance checking problem as we do, and with a more efficient time complexity algorithm than we have proposed in our work.
In [18] the authors take a somewhat different direction. They show how to synthesize reactive systems, given traces of input/output symbols that represent the reactive behavior of the system specification.

Reactive Models with Infinite Memory
In this section we present the Visibly Pushdown Labeled Transition System (VPTS) and introduce its variation, the Input/Output VPTS. But first we establish some notation that will be useful.

Basic Notation
Let X and Y be sets. We indicate by P(X) = {Z | Z ⊆ X} the power set of X, and X−Y = {z | z ∈ X, z ∈ Y } indicates set difference. An alphabet is any non-empty set of symbols. Let A be an alphabet. A word over A is any finite sequence σ = x 1 . . . x n of symbols in A, that is, n ≥ 0 and x i ∈ A for all i = 1, 2, . . . , n. When n = 0, σ is the empty sequence, also indicated by ε. The set of all finite sequences, or words, over A is denoted by A , and the set of all nonempty finite words over A is indicated by A + . When we write x 1 x 2 . . . x n ∈ A , it is implicitly assumed that n ≥ 0 and that x i ∈ A, 1 ≤ i ≤ n, unless noted otherwise. The length of a word α over A is indicated by |α|. Let σ = σ 1 . . . σ n and ρ = ρ 1 . . . ρ m be words over A. The concatenation of σ and ρ, indicated by σρ, is the word σ 1 . . . σ n ρ 1 . . . ρ m . A language G over A is any set We will also need the notion of a morphism between alphabets. Let A, B be alphabets, a homomorphism, or just a morphism, from A to B is a function h : A → B . Such a morphism can be inductively extended to a function h : A → B , thus: We can further lift h to a function h : P(A ) → P(B ) by letting h(G) = σ∈G h(σ), when G ⊆ A . We may write h instead of h, or of h, when no confusion can arise. When a ∈ A, we let h a : A → A − {a} be the morphism where h a (a) = ε, and h a (x) = x when x = a. Hence, h a (σ) erases all occurrences of a in the word σ.

Visibly Pushdown Labeled Transition Systems
A Labeled Transition System (LTS) is a formalism convenient to express an asynchronous exchange of messages between participating entities, in the sense that outputs do not have to occur synchronously with inputs, but are generated as separated events. Any LTS, however, has only a finite memory, represented by its set of states. A Visibly Pushdown Labeled Transition System (VPTS), on the other hand, has a pushdown memory associated to it, and thus can make use of a potentially infinite memory. The next definition is inspired by the notion of a Visibly Pushdown Automaton [7]. -S in ⊆ S is the set of initial states; -L is a set of labels, or action symbols,, partitioned thus L = L c ∪ L r ∪ L i ; -There is a special symbol ς / ∈ L, the internal action symbol; -Γ is a set of pushdown symbols. There is a special symbol ⊥ ∈ Γ, the bottom-of-stack symbol; The class of VPTSs over a set of labels L will be indicated by VP(L).
Let t = (p, x, Z, q) ∈ T . If t ∈ T c we say that it is a push-transition. Its intended meaning is that S, in state p ∈ S and reading the push symbol x, changes to state q and pushes Z onto the stack. When t ∈ T r we have a pop-transition, with the intended meaning that, in state p ∈ S, reading the pop symbol x ∈ L r and having Z as the topmost symbol in the stack, S pops Z from the the stack and changes to state q. Further, when the stack is reduced to the bottom of stack symbol, ⊥, then a pop move can be taken, leaving the stack unchanged. We have a simple-transition when t ∈ T i and x ∈ L i , and we have an internal-transition when t ∈ T i and x = ς. The meaning of a simple-transition t is to change from state p to state q, while reading the simple symbol x and leaving the stack unchanged. An internal-transition also changes from state p to state q leaving the stack unchanged, but reads no symbol from the input. The following definition makes these notions precise.
When p ∈ S in and α = ⊥, (p, α) is an initial configuration. The set of all configurations of S is indicated by C S .
We call (p, α) → (q, β) an elementary move of S, and we say that (p, , Z, q) is the transition used in the move (p, α) → (q, β).
It is clear that after any elementary move (p, α) → (q, β) we have (q, β) ∈ C S , that is, (q, β) is also a configuration of S. Hence, → induces a binary relation on C S .

Remark 1
In figures, a push-transition (s, x, Z, q) will be graphically represented by x/Z + next to the corresponding arc from s to q. Similarly, the label x/Z − will indicate a pop-transition (s, x, Z, q). A simple-or internal-transition (s, x, , q) will be indicated by the label x next to the corresponding arc.
2. Let σ ∈ L . We say that σ is an observable path from (p, α) to (q, β) in S if there is a path µ from (p, α) to (q, β) in S such that σ = h ς (µ).
In both cases we say that the path starts at (p, α) and ends at (q, β), and we say that the configuration (q, β) is reachable from (p, α). We also say that (q, β) is reachable in S if it is reachable from an initial configuration of S. Moves labeled by the internal symbol ς can occur in a path. An observable path is just a path from which ς-labels were removed. If σ is a path from (p, α) to (q, β), this can also be indicated by writing (p, α) σ → (q, β). When |σ| = 1 this has exactly the same meaning as indicated in Definition 2, so that no confusion can arise. We may also write (p, α) σ → to indicate that there is some (q, β) ∈ C S such that (p, α) σ → (q, β); likewise, (p, α) → (q, β) means that there is some σ ∈ (L ∪ {ς}) * such that (p, α) σ → (q, β). Also (p, α) → means (p, α) σ → (q, β) for some (q, β) ∈ C S and some σ ∈ (L ∪ {ς}) * . When σ is an observable path from (p, α) to (q, β) we may write (p, α) σ ⇒ (q, β), with similar shorthand notation also carrying over to the ⇒ relation. When we want to emphasize that the underlying VPTS is S, we write (p, α) Paths starting at (p, α) are also called the traces of (p, α), or the traces starting at (p, α). The semantics of a VPTS is related to traces starting at an initial configuration.
2. The semantics of S is tr(S) = q∈Sin tr(q, ⊥), and the observable semantics of S is otr(S) = q∈Sin otr(q, ⊥).
We can restrict the syntactic description of VPTS models somewhat, without loss of generality, by removing states that are not reachable from any initial state, since these states can not affect the system behavior. Moreover, we can also eliminate ς-labeled self-loops. We formalize these observations in the following hypothesis.
Hypothesis 1 Let S, S in , L, Γ, T ∈ VP(L). For any s ∈ S there is some α ∈ Γ and s 0 ∈ S in such that We see that a VPTS can autonomously move along ς-transitions, without consuming any input symbol. However, in some situations such moves may not be desirable, or simply we might want no observable behavior leading to two distinct states. This motivates the notion of determinism.
As a consequence, deterministic VPTSs have no internal moves.
Proof By contradiction, assume that (s, ς, , q) ∈ T . From Remark 1 we get s = q and we also get α ∈ Γ , . Since s = q, this contradicts Definition 5.

The Product of two VPTSs
It captures the synchronous behavior of the two models, and will be useful when testing conformance between two VPTS models.
The following result links moves in the original VPTSs to their product.
On the other direction, we have a similar result.

Input Output Pushdown Transition Systems
The VPTS formalism can be used to model systems with a potentially infinite memory and with a capacity to interact asynchronously with an external environment. In such situations, we may want to treat some labels as symbols that the VPTS "receives" from the environment, and some other labels as symbols that the VPTS "sends back" to the environment. The next VPTS variation captures this idea.

Remark 2
In order to keep the number of definitions under control, we agree that in any reference to a notion based on IOVPTSs, and that has not been explicitly defined at some point, we substitute the IOVPTS model by its underlying VPTS.
The semantics of an IOVPTS is just the set of its observable traces, that is, the observable traces of its underlying VPTS. where S is the underlying VPTS associated to I. Figure 1 can be seen as an IOVPTS that describes a simple drink dispensing machine. In this case we have L I = {b} and L U = {c, t}. From the context we can see that L c = {b}, L r = {c, t} and L i = ∅. The start state is s 0 . Symbol b stands for button an user can press when asking for a cup of coffee or a cup of tea, with corresponding buttons represented by the labels c and t, respectively. Each time b button is activated, the model pushes the symbol Z on the stack, so that the stack is used to count how many times the b button was hit by the user.

Example 2
At any instant, after the user has activated the b button at least once, the machine moves to state s 1 and starts dispensing either coffee or tea, indicated by the c and t buttons. It decrements the stack each time a drink is dispensed, so that it will never deliver more drinks than the user asked for.
A move back to state s 0 , over the internal label ς interrupts the delivery of drinks, so that the user can, possibly, receive less drinks than originally asked for. In this case, when the next user operates the machine it is possible, eventually, to collect more drinks than asked for. An alternative model could use one more state s 2 to interrupt the transition from s 1 to s 0 and install a self-loop at s 2 that empties the stack. A more realistic drink dispensing machine is illustrated in Subsection 5.2. 2 We register one more example which will be used later.

Conformance Relation and Fault Models
In this section we provide a method to check whether IUTs, described as IOVPTSs, conform to a given specification IOVPTS model. We define an ioco-like conformance relation for IOVPTSs. The idea is that, given a specification S and an IUT I, we say that I conforms to S when, for any observable behavior σ of S, any output symbol that I can emit after running over σ is, necessarily, among the output symbols that S can also emit after it runs over the same σ. That notion captures the same behavior as the standard notion of ioco-conformance [3,6] for LTS models, but the latter do not have access to an auxiliary pushdown store, as IOVPTSs models do.

An ioco Conformance Relation for IOVPTS Models
Let S be a specification and I an IUT. The ioco-like relation essentially says that if σ leads I to a configuration from which it can emit the output , then this must also hold for S.
Proof Immediate from Lemma 1.
Example 5 Let the IOVPTS S of Figure 2 be the specification, with L I = {a, b}, and L U = {x}. We want to construct a model for a test suite T ⊆ (L i ∪ L U ) that can be used for testing whether I ioco-like S, for any implementation I. From Corollary 1, we know that I ioco-like S is equivalent to otr(I ∩ T ) = ∅, where T = otr(S) ∩ D, where D = otr(S)L U . So, we want a model that describes such a T . We start with a representation for D = otr(S)L U . Consider the VPTS TS depicted in Figure 4. Note that x is the only symbol in L U . TS was obtained from S by adding all possible transitions over x to the new state d of TS, starting at all states of S from which that transition was not present in S. Hence, it is clear that D = otr(S)L U is the set of all σ that lead TS from the initial configuration (d 0 , ⊥) to a configuration (d, α⊥). Also, note that all transitions into state d 0 add a symbol A to the stack. Hence, S can never make a move on the pop symbol x from state s 0 when the stack is empty. The other possible moves of TS into d from states d 1 and d 2 obviously are not possible moves in S. We conclude that all elements in D are also in otr(S). This gives T = otr(S) ∩ D = D.
Thus, to verify if an IUT I ioco-like-conforms to S, we must check if there is some element in D that is also in the observable semantic of I. In the next two subsections we use the notion of a fault model to make this procedure more systematic.

IOVPTS Fault Models
First, we formally model the external environment as an IOVPTS with a special fail state. Such a model, T, operates in conjunction with an IUT, I. Their joint behavior can be interpreted as T sending symbols to I, and I responding by sending symbols back to T. Note that, in this setting, the sets of input and output symbols in T and I must be interchanged.
Definition 10 Let L I and L U be sets of input and output symbols, respectively. An Input/Output Visibly Pushdown Fault Model ( IOVPFM) is any T ∈ IOVP(L U , L I ) with a distinguished fail state.
Given an IOVPFM T and an IUT I, the exchange of action symbols between T and I can be described by the product of their underlying VPTSs. Recall Definition 6.
Definition 11 Let S, I ∈ IOVP(L I , L U ) and L = L I ∪ L U . Their product is V S × V I , with V S , V I ∈ VP(L) being the underlying VPTSs of S and I, respectively.
In order to facilitate the notation, we will also denote the cross-product of S and I simply by S × I. Having an implementation I and an IOVPFM T as a tester, we need to say when a test run is successful with respect to a given specification S. Recalling that T signals an unsuccessful run when it reaches a fail state, we will say that an IUT I passes T when no synchronous execution of T and I reaches a (fail, q) state in T × I. In this case, we want I ioco-like S to hold. Alternatively, if some run of T × I does reach a (fail, q) state, that is, if I does not pass T, then we want a guarantee that I ioco-like S does not hold. In other words, we need a property of completeness.
Definition 12 Let T = Q, Q in , L U , L I , ∆, R ∈ IOVP(L U , L I ), and I = S, S in , L I , L U , Γ, T ∈ IOVP(L I , L U ). We say that I passes T if, for all σ ∈ (L I ∪ L U ) and all initial configurations ((t 0 , q 0 , q), α⊥), for any configuration ((fail, q), α⊥) of T × I. Also let S ∈ IOVP(L I , L U ). We say that T is ioco-like complete for S if we have I ioco-like S if and only if I passes T, for all I ∈ IOVP(L I , L U ). Now we construct an IOVPFM which is complete for a given specification S.
Lemma 2 Let S ∈ IOVP(L I , L U ) be deterministic with n states. We can effectively construct an IOVPFM T which is ioco-like complete for S. Moreover, T is deterministic and has n + 1 states.
Proof According to Definition 12, in order for T to be ioco-like complete for S we need that, for all implementations I, it holds that I passes T if and only if I ioco-like S. From Corollary 1 we know that Since S has n states, it is clear that T has n+1 states and a single fail state. Further, using Proposition 1, we see that S has no ς-moves. Hence, by construction T has no ς-moves.
According to Definition 5 we want to show that s 1 = s 2 and α 1 = α 2 . Since T has no ς-moves, we can write If p 1 = fail = p 2 then, since fail is a sink state, we must have (s i , ⊥) σ → S (p i , α i ⊥), i = 1, 2. The determinism of S gives the desired result in this case. Now assume p 1 = fail, p 2 = fail. Since s 1 ∈ S in we get s 1 = p 1 so that σ = µx with x ∈ L I ∪ L U , and we can write (s i , ⊥) The determinism of S implies r 1 = r 2 and β 1 = β 2 . So, we have (r 1 , x, Z 1 , fail) in T and (r 2 , x, Z 2 , p 2 ) = (r 1 , x, Z 2 , p 2 ) in T, for some Z 1 , Z 2 ∈ ∆ T . Clearly, (r 1 , x, Z 2 , p 2 ) is S. If x is a push symbol, then Z 1 = Z. Now, (r 1 , x, Z 2 , p 2 ) in S and (r 1 , x, Z, fail) in T contradict Eq. (1). Similarly when x is a simple symbol we get a contradiction to Eq. (3). When x is a pop symbol, let W be the first symbol in β 1 ⊥ = β 2 ⊥. We now have Z 1 = W = Z 2 and then (r 1 , x, W, p 2 ) in S and (r 1 , x, W, fail) in T contradict to Eq. (2).
We conclude that T is deterministic.
The next claim shows that any σ ∈ otr(S) ∩ otr(S)L U leads T to the fail state.
We now argue that (p, α⊥) → as needed. We note that we cannot have (p, α⊥) → S (z, γ⊥) for any z ∈ S S , γ ∈ (∆ S ) , because then we would get (s 0 , ⊥) µ → S (z, γ⊥), and then µ = σ ∈ otr(S), a contradiction. There are three simple cases. If ∈ L c , then (p, , W, z) ∈ T S for any z ∈ S S and any W ∈ ∆ S . Then, Eq. (1) gives (p, , Z, fail) ∈ T T , as needed. If ∈ L i , the reasoning is the same, using Eq. (3). Now let ∈ L r . Since The next claim deals with the converse.
Proof. Since there are no ς-moves in T, we must have (t 0 , ⊥) Since fail is a sink state we get p = fail, and we know that all transitions in (t 0 , ⊥) α⊥). Thus, µ ∈ otr(S). We must also have a transition (p, , X, fail) in T. By the construction, it can only be inserted in T T by force of Equations (1),(2), or (2). In any case, we get ∈ L U . Hence σ = µ ∈ otr(S)L U .
Assume that σ ∈ otr(S). Since S has no ς-moves we get (s 0 , ⊥) Since (p, , X, fail) is a transition of T, together with (p, α⊥) → S (r, γ⊥), there are three cases. We show that all lead to a contradiction, thus showing that σ ∈ otr(S) as desired.
If ∈ L c then (p, , W, r) is a transition of S for some W ∈ ∆ S . In this case we must have used Eq. (1) to insert (p, , X, fail) in T T . But then we need X = Z and (p, , Y, q) ∈ T S for any q ∈ S S , Y ∈ ∆ S , and we get a contradiction. If ∈ L i then W = , (p, , , r) is a transition of S and we must have used Eq. (3) to insert (p, , , fail) in T T . But this requires (p, , , q) ∈ T S for any q ∈ S S , and we reach a contradiction again. If ∈ L r then (p, , W, r) is a transition of S where W is the first symbol in α⊥. According to used Eq. (2) we now need (p, , W, q) ∈ T S for any q ∈ S S . Let I = S I , I in , L I , L U , ∆ I , T I be an arbitrary IUT. As argued above, we need I does not pass T if and only if otr(I) ∩ T = ∅, where T = otr(S) ∩ otr(S)L U . According to Definition 12, I does not pass T if and only if for some σ ∈ (L I ∪ L U ) we get ((t 0 , q 0 ), ⊥) σ ⇒ T×I ((fail, q), α⊥) where ((t 0 , q 0 ), ⊥) is an initial configuration of T × I, q ∈ S I and α = (X 1 , Y 1 ) · · · (X n , Y n ) ∈ (∆ T × ∆ I ) , n ≥ 0. Since fail is a sink state and transitions into fail are over symbols in L U , we can say that I does not pass T if and only if for some µ ∈ L , ∈ L U , p ∈ S T , p = fail, r ∈ S I , β = (W 1 , Z 1 ) · · · (W m , Z m ) ∈ (∆ T × ∆ I ) , m ≥ 0.

Testing IOVPTS Models for ioco-like conformance
Given an IOVPFM which is complete for a given specification, we can test whether IUTs ioco-like conform to that specification. But first we define the notion of balanced run.
Let V ∈ VP(L), and let p, q be states of V. We say that a string σ ∈ L induces a balanced run from p to q in V if we have (p, ⊥) σ → V (q, ⊥). The next theorem gives a decision procedure for testing ioco-like conformance.
Theorem 6 Let S = S S , {s 0 }, L I , L U , ∆ S , T S ∈ IOVP(L I , L U ) be a deterministic specification, and let I = S I , I in , L I , L U , ∆ I , T I ∈ IOVP(L I , L U ) be an IUT. Then we can effectively decide whether I ioco-like S holds. Further, if I ioco-like S does not hold, we can find σ ∈ otr(S), ∈ L U that verify this condition, i.e., ∈ out((q 0 , ⊥) after σ) for some q 0 ∈ I in , and ∈ out((s 0 , ⊥) after σ).  , q), α⊥) for some σ ∈ L , where P = T × I is the product IOVPTS, and (t 0 , q 0 ) is an initial state of P. So, in order to check for ioco-like conformance it suffices to check whether a configuration ((fail, q), α⊥) is reachable from some initial configuration of P. First, we modify P in a simple way in order to make this reachability problem more amenable.

Proof
Emptying the stack after reaching a state (fail, q). For all states (fail, q), add the internal transition ((fail, q), ς, , f 1 ) to P, where f 1 is a new state. Then, for all stack symbols W add the self-loops (f 1 , b 1 , W, f 1 ) to P, where b 1 is a new pop symbol added to L. Next, add the transition (f 1 , b 1 , ⊥, f 2 ) to P, where f 2 is another new state. Let P 1 be the resulting IOVPTS obtained after these modifications to P. Since (fail, q) is a sink state in P, it is easy to see that (1) If ((t 0 , q 0 ), ⊥) σ → ((fail, q), α⊥) in P then ((t 0 , q 0 ), ⊥) µ → (f 2 , ⊥) in P 1 , where µ = σςb k 1 with k = |α| + 1.
Assume that P has been transformed as described, so that we can always empty the stack after reaching a (fail, q) state, for all states q of I.
Eliminating pop moves on an empty stack. Let s 0 be yet a new state, a 2 a new push symbol, and Z 2 a new stack symbol. Add the self-loop (s 0 , a 2 , Z 2 , s 0 ) to P. Next we connect s 0 to all original initial states of P with internal transitions (s 0 , ς, , s) where s is initial P, and make s 0 the new, unique, initial symbol of P. Finally, we replace any pop transition on an empty stack (p, c, ⊥, q) by the pop transition (p, c, Z 2 , q). Let P 2 be the new IOVPTS after these modifications to P. Now we have (1) If (s, ⊥) σ → (q, ⊥) in P where s is one of its initial state, and if we have 0 ≤ k ≤ |σ| pop moves on the empty stack on this run, then by an induction on k we can show that (s 0 , ⊥) (2) If (s 0 , ⊥) σ → (q, ⊥) in P 2 , then an easy argument shows that σ = a k 2 ςµ, with k ≥ 0, and we have (s, ⊥) µ → (q, ⊥) in P, where s is the initial state in P and this run makes k pop moves on an empty stack.
Assume that the original product P = T × I has been transformed into the IOVPTS P after the modifications that allow us to empty the stack after reaching a (fail, q) state and to avoid pop moves on an empty stack. Then, we have ((t 0 , q 0 ), ⊥) We have reduced the ioco-like conformance test to the following problem: given two states p and q of an IOVPTS, find a string σ that induces a balanced run from p to q, or indicate that such a string does not exist. Next, we solve this problem.
The following construction was inspired from ideas of previous works [14,19]. Now, it is sufficient to consider the underlying VPTS. So, let P = Q, Q in , L, Γ, ρ be a VPTS given by an incidence vector P of transitions, indexed by Q, where P [p] points to a list of all transitions (p, x, Z, q) ∈ ρ where p is the source state. We assume that P has no pop transitions on the empty stack, that is, of the form (p, x, ⊥, q) where x is a pop symbol. Algorithm 1 shows the pseudo-code.  We will use two vectors of pointers, In and Out, both indexed by Q, and a queue V . The entry In[p] will point to a list of triples (s, a, Z) corresponding to transitions (s, a, Z, p) where a ∈ L c , that is, p as the target state of a push transition. Likewise, an entry in Out[p] will point to a list of triples (a, Z, s) corresponding to transitions (p, a, Z, s) where a ∈ L r , that is, p is the source state of a pop transition. We will also need a square matrix R, indexed by Q × Q, where R[p, q] will contain: (i) [a, p, q, b], or (ii) [p, c, q], or (iii) [p, s, q], or (iv) 0, where a ∈ L c , b ∈ L r , c ∈ L i ∪ {ς}, and p, q, s are states. The general idea is that, when R[p, q] = 0 then it will code for a string σ that induces a balanced run from p to q.
We now examine Algorithm 1. Lines 1-10 initialize V , In, Out and R. At line 6, note that a transition (p, a, , q) immediately induces a balanced run from p to q. At line 10, we collect a simple balanced run from p to r that is induced by a push transition (p, a, Z, q) and a pop transition (q, b, Z, r). In the main loop, lines 11-19, removing (p, q) from V indicates that we already have a string, say σ, that induces a balanced run from p to q. At lines 13-14, a string µ that induces balanced run from s to p is encoded in R[s, p]. Hence, µσ induces a balanced run from s to q. If we still do not have a balanced run from s to q, we can now encode the string µσ in R[s, q] and move the pair(s, q) to V so that it can be examined later. Lines 15-16 do the same, but now we encode in R[p, t] a string that induces a balanced run from p to t. At lines 17-19, we search for a push transition (s, a, Z, p) and a matching pop transition (q, b, Z, t) and, when successful, we encode aσb as a string that induces a balanced run from s to t. The cycle repeats until saturation when V = null, or until we find a balanced run from s i to s e , as requested. Lines 22-26 list a recursive procedure that extracts the string encoded in R[p, q] = 0.

Correctness and Complexity
First we argue for correctness.
Theorem 7 Let P = Q, Q in , L, Γ, ρ be a VPTS with no transitions of the form (p, a, ⊥, q) in ρ. Also let s i , s e ∈ Q, with s i = s e . Suppose P, s i , s e are input to Algorithm 1. Then it stops and returns a string σ ∈ L such that (s i , ⊥) σ → P (s e , ⊥), or it indicates that such a string does not exist.
Proof A VPTS P = Q, Q in , L, Γ, ρ with no transitions of the form (p, a, ⊥, q) in ρ, and two states s i , s e with s i = s e , are input to Algorithm 1.
At lines 1 and 2 we start with V = null and R[p, q] = 0 for all pairs (p, q). Inspecting lines 6, 10, 14, 16 and 19, we see that a pair (p, q) is added to V only when we currently have R[p, q] = 0 and, when(p, q) enters V we immediately set R[p, q] to some nonzero value. Further, at no other point in the main loop, at lines 11-19, we reset R[p, q] to zero. Hence, a pair (p, q) can enter V at most once and, therefore, the main loop at line 11 must terminate. So, Algorithm 1 always stops.
Next we claim that, at any point during the execution of the algorithm, if R[p, q] = 0 then it codes for a string that induces a balanced run from p to q. This is immediate from the initialization lines 6 and 10. Proceeding inductively, assume that this property holds after a number of executions of the main loop. At line 14, we have removed (p, q) from V , and so we now have R[p, q] = 0 because (p, q) entered V in a previous iteration. At that moment we made R[p, q] = 0 and then, inductively, it codes for a string σ such that (p, ⊥) for some σ ∈ (L ∪ {ς}) . Recall that the algorithm assumes that the given VPTS P has no pop transitions on the empty stack. With this hypothesis, we claim the following general property of P: If (p, α 1 ⊥) σ → P (q, α 2 ⊥) with p, q ∈ Q and α 1 , α 2 ∈ Γ , then for all β 1 , β 2 ∈ Γ we also have A simple proof can be obtained by induction on |σ| ≥ 0. ⊥), so that making R[s, t] code for the string aσb also extends the induction after line 19 is passed. Since we have completed one more iteration of the main loop, we see that upon termination of the main loop, if we have R[s i , s e ] = 0, then we do have a string that induces a balanced run from s i to s e . Moreover, it easy to see that the simple recursive call getstring(s i , s e ) at line 21 does correctly extract one such string.
Next, we argue in the other direction. Suppose that the main loop terminates with V = null. Then we claim that for all pairs (p, q), with p = q, if R[p, q] = 0 then there is no string capable of inducing a balanced run from p to q. For the sake of contradiction, assume that the main loop terminates with V = null, and we have p = q, R[p, q] = 0, and a string σ such that (p, ⊥, ) σ → (q, ⊥). Among all such pairs, choose one for which |σ| is minimum. Since p = q, we need |σ| ≥ 1. If |σ| = 1, then we need a transition (p, σ, , q) in ρ. But then, at line 6, we make R[p, q] = [p, σ, q] and it is never rest to 0 again. This is a contradiction, and we can assume |σ| ≥ 2. Now there are two cases, depending on the configurations that occur between (p, ⊥) and (q, ⊥) in the run (p, ⊥) σ → (q, ⊥): → (q, ⊥), with σ = σ 1 σ 2 . We know that σ 1 = ε = σ 2 because P has no transitions on the empty stack. If p = r we get (p, ⊥) σ2 → (q, ⊥). Since |σ 2 | < |σ|, the minimality of |σ| forces R[p, q] = 0, a contradiction. Similarly, q = r also leads to a contradiction. Now, assume p = r = q. Since |σ 1 | < |σ| and |σ 2 | < |σ|, when the main loop terminates with V = null we must have R[p, r] = 0 and R[r, q] = 0. Moreover, for this to happen, both (p, r) and (r, q) were added to V . Suppose that (p, r) is removed from V before (r, q). Then, at the iteration of the main loop when (r, q) is removed from V we have R[p, r] = 0 and p = q. Hence, at line 14, since R[p, q] = 0, we make R[p, q] = [p, r, q] and, since it is never reset to 0 again, we have a contradiction. If (r, q) is removed first from V , the reasoning is the same using line 16. So, this case can not happen. Next we claim that in any VPTS S = S, S in , L, Γ, T if a run does not shorten the initial stack, then that stack can be replaced by any other. More precisely, Let p, q ∈ S, σ ∈ (L∪{ς}) and α ∈ Γ with (p, α⊥) σ → (q, α⊥). Assume that a configuration (u, γ), with |γ| < |α|, does not occur in that run over σ. Then, for any β ∈ Γ we also have (p, β⊥) σ → (q, β⊥).
An easy induction over |σ| ≥ 0 gives the result.
Recall that we already have (s, Z⊥) µ → (t, Z⊥) and a configuration (u, ⊥) dos not occur on the run over µ since it can not occur on a run over σ. Using the claim we get (s, ⊥) µ → (t, ⊥). Now, since |µ| < |σ|, the minimality of |σ| says that when the main loop terminates with V = null, we must have R[s, t] = 0. But then, at some moment (s, t) was added to V . Since the main loop terminates with V = null, at some iteration we have removed (s, t) from V . Note that we have a push transition (p, x, Z, s) and a pop transition (t, y, Z, q). Hence, line 19 says that we will set R[p, q] = [x, s, t, y] and, since R[p, q] is never reset, we see that the main loop terminates with R[p, q] = 0, which is a contradiction.
Since both cases lead to contradictions, we get that when the main loop terminates with V = null and R[s i , s e ] = 0, then there is no string capable of inducing a balanced run from s i to s e . Hence, line 20 correctly reports the inexistence of any such strings.
Thus, lines 20-21 always report as expected, and Algorithm 1 is correct. Now we examine the complexity of our testing approach. Proof Write L = L I ∪ L U , |L| = and let g s , g i be the number of stack symbols in S and I, respectively. We now follow the argument in the proof of Theorem 6.
First, the fault model T is constructed in Lemma 2. From Equations (1)-(3) we see that n t = n s + 1 and m t ≤ m s + n s g s , where n t and m t are the number of states and transitions in T, respectively. It is easy to see that T can be effectively constructed from S by an algorithm with worst case time complexity bounded by O(m s + n s g s ). We can safely assume m s ≥ g s and m s ≥ . Hence, m t and n t can be bounded by O(n s m 2 s ) and O(n s ), respectively, and the worst case time complexity to obtain T can also be bounded by O(n s m 2 s ). Next, we construct the product P = T × I. Let n p , m p and g p be the number of states, transitions and stack symbols in P, respectively. Using Definition 6, and since T has no ς-moves, we see that n p = n t n i , m p ≤ m t m i + n t m i , and g p = g s g i . As before, a simple algorithm with worst case time complexity Finally, Theorem 6 requires that we modify P to an IOVPTS with an underlying VPTS A = S a , {s 0 }, L a , Γ a , ρ a with the property that I ioco-like S does not hold if and only if we have (s 0 , ⊥) µ → A (f, ⊥) for some µ ∈ (L a ∪ {ς}) , where f is a specific state in S a with s 0 = f . Moreover, if such is the case, the final steps in the proof of Theorem 6 indicate how to obtain the desired string σ that proves that I ioco-like S fails. Let n a = |S a | and m a = |ρ a |. From the proof of Theorem 6, it is easy to get n a = n p + 3 and m a = m p + 2n i + g p + 2. Also, a simple procedure, running in worst case time complexity O(m a + n a ), can be used construct the VPTS A given the product P. Then, n a can be bounded by O(n s n i ), m a can be bounded by O(n s m 2 s m i ), and the worst case time complexity to construct A can be bounded by O(n s m 2 s m i ). The final step is to submit A and the two states s 0 , f to Algorithm 1. Theorem 7 guarantees that Algorithm 1 correctly produces a desired string or indicates that no such string exists.
We now derive an asymptotic upper bound on the number of steps required for Algorithm 1 in the worst case. Clearly, the number of steps for lines 1-3 can be bounded by O(n 2 a ). For each state p ∈ S a , let s p , t q be the number of transitions in A that have p as a source and q as a target state, respectively. Thus, p∈Q s p ≤ m a and p∈Q t p ≤ m a for all p ∈ S a . The total number of steps pertaining to lines 4-10 can be bound by From the proof of Theorem 7, we have that each pair (p, q) can enter the queue V at most once. Hence, a state p will appear in a pair (p, q) removed from V at most n a times. Since the number of steps at each execution of lines 13-14 can be bounded by O(n 2 a ), the total effort spent for lines 13-14 is bound by O(n 3 a ). Likewise for lines 15-16. Now we bound the total number of steps for lines [17][18][19]. For each pair (p, q) removed from V , the cost relative to lines 17-19 is O(t p s q ). Since each pair of (p, q) can enter V at most once, the total cost is bound by Hence, the total number of steps to execute the main loop at lines 12-19 is bound by O(n 3 a + m 2 a ). We can now conclude that the total number of steps to execute Algorithm 1 is bound by O(n 3 a + m 2 a ). Using the previously computed values, we see that a worst case asymptotic time complexity for Algorithm 1 is bounded by O(n 3 s n 3 i + n 2 s m 4 s m 2 i ). Since this bound dominates all the preprocessing steps needed to construct T, P and A, the overall worst case time complexity of the ioco-like checking procedure is O(n 3 s n 3 i + n 2 s m 4 s m 2 i ).
In some practical situations we may assume that the number of stack and alphabet symbols as constants, for any specifications and IUTs models that will be considered. In these cases, the number of transitions of the fault model T can be bounded by O(m s ), in the proof of Theorem 8. As a consequence, the worst case time complexity for Algorithm 1 can be seen to be bounded by O(n 3 s n 3 i + m 2 s m 2 i ).

Example: a Drink Dispensing Machine
Now we want to apply the previous results in a more realistic setting where a drink dispensing machine is operated. Because the overall testing procedure is not yet fully implemented on software, the example has to be somewhat contrived, so that we can proceed manually. So we show only some possible interactions that may occur. When dealing with a practical implementation, of course, many other situations could be represented, making the model as complex as required by the testing requirements one is dealing with. We first describe the drink dispensing machine and its specification IOVPTS model. In the sequel we construct the fault model for the given specification and then we test some possible IUTs for ioco-like conformance.

A Drink Dispensing Machine
In a typical drink dispensing machine, a customer puts in some money and then order the desire beverage. After choosing the beverage, the right amount of money will be charged and the machine will dispense the chosen drink. If the amount of money was in excess, the customer can ask for the balance. If the amount of money already in the machine is not enough, the customer has to add more money or the customer can decide to get a full refund. Usually, real machines accept several payment methods such as cash and credit cards. In order to ease our modeling we specify that only unit coins can be used for payment.
The complete IOVPTS specification model S = S S , S in , L I , L U , ∆ S , T S is depicted in Figure 5, where L I = {coi, rch, crd, wtr, tea, cof, deb} is the set of input events and L U = {chg, dwt, dte, dco} is the set of We have split state s 1 to make the figure clearer. Recall Remark 1 for the notation. The underlying VPTS is The system starts at state s 1 where the customer can either insert coins into the machine -event labeled coi -, request his change -event labeled rch -, or ask for a drink, namely, label wtr for water, label tea for tea, or label cof for coffee. Inserting coins is represented by the self-loop labeled coi/C + at state s 1 . Note that the pushdown stack keeps track of the number of coins inserted into the machine. At state s 1 the customer can also request a refund, or the remaining change, by activating the rch event. The machine will then return the correct balance via the pop self-loop crd/C − at state s 3 .
The customer orders a drink by pushing the button for water, tea, or coffee, moving the machine to states s 2 , s 4 and s 6 , respectively. The price associated to water is one coin, for tea it is two coins and for coffee it is three coins. When the customer asks for water, the pop transition dwt/C − is taken, returning to state s 1 , and the correct charge is applied subtracting one coin from the total amount. The event dwt indicates that water has been dispensed. However, if not enough coins have been inserted, the transition from state s 2 back to state s 1 is blocked. The customer can proceed by inserting more coins using the self-loop coi/C + at state s 2 . The behavior when ordering tea or coffee follows similar paths. For simplicity, once the customer has made a commit to order some of the beverages, the machine will wait until enough coins have been inserted to pay for the chosen drink. Figure 6 depicts the fault model T that is constructed for the specification S using Lemma 2. We have split the fail state in order keep the figure uncluttered. The sets D , for ∈ {a, r, w, t, c}, collect the label of several transitions, as indicated in the figure caption. Note that L U ∩ L c = ∅ = L U ∩ L i , so that the only transitions into the fail state we need to check are those over the symbols of L U ∩ L r = {chg, dwt, dte, dco} together with stack symbols in {C, ⊥}. For example, the set D a denotes transitions to the fail state with pairs (x, y) for all x ∈ L U ∩ L r and all y ∈ {C, ⊥}.

Testing Some Implementations
In this subsection, we examine some implementations, and test them for ioco-like conformance against the specification S depicted in Figure 5, and whose fault model T is shown in Figure 6.
Our first example is an IUT I a , depicted in Figure 7, where coffee is wrongly charged at two coins only. Here we notice that the state s 8 of S is missing, so that in the IUT I a we have a self-loop (s 7 , deb/C − , s 7 ) instead of the transition (s 7 , deb/C − , s 8 ) as in the specification model. In this case a fault can occur and the machine may charge less for a cup of coffee.
Consider the sequence of events η = µ dco, where µ = coi coi cof deb. The customer has inserted only two coins and ordered coffee, and still the machine may deliver a cup of coffee. It is easy to see that η leads T to the fail state, while I a reaches state s 1 . Again, from Lemma 2 we obtain that I a ioco-like S does not hold. In this example we have dco ∈ L U and dco ∈ out((s 1 , ⊥) after µ) in I a , while dco ∈ out((s 1 , ⊥) after µ) in S. So, from Definition 9, we can declare that I a ioco-like S does not hold. In the product T × I a , we get the corresponding run ((s 1 , s 1 ), ⊥) η ⇒ ((fail, s 1 ), ⊥).
Notice that, in this same IUT I a , coffee could be charged more than three coins, i.e., the machine may subtract more than three coins before dispensing a cup of coffee, when the user has inserted more than three coins before asking for the cup of coffee. Now, we turn to IUT I b , obtained from S by adding the extra self-loop (s 5 , deb/C − , s 5 ) to Figure 5. This allows the machine to subtract any number of extra coins after the customer has ordered a drink, given that more than enough coins have been inserted. Consider the sequence of events coi coi coi tea deb deb dte rch chg, signaling that the customer initially has inserted three coins, then decided to order tea. According to the IUT I b , however, when requesting the remaining change the customer gets no coins back, and the net effect was that the customer was charged three coins for a cup of tea. However, even in face of that mistake, we show below that I b does conform to the specification S. Recall the original specification S. We note that I b differs from S only by the extra transition at state s 5 . Further, for each symbol x ∈ L U and state s i , there is at most one transition out of s i on x, both in I b and in S. Reasoning more formally, it is easy to see that for any sequence of events σ a simple induction on |σ| ≥ 0 shows that if we have (s i , α⊥) ∈ (s 1 , ⊥) after σ in S and (s j , β⊥) ∈ (s 1 , ⊥) after σ in I b , then i = j and α = β. According to Definition 9, if I b ioco-like S did not hold, we would need ∈ L U and a sequence σ such that ∈ out((s i , α⊥)) where (s i , α⊥) ∈ (s 1 , ⊥) after σ in I b and ∈ out((s j , β⊥)) where (s j , β⊥) ∈ (s 1 , ⊥) after σ in S. Since the transitions of S and I b are identical, except at state s 5 , we conclude that s i = s j = s 5 and = deb. Because deb ∈ L U we reached a contradiction, and must conclude that I b ioco-like S does hold.
We note that deb ∈ L U was crucial to the preceding argument. In fact, if we move deb from L I to L U , then we clearly would get that I b ioco-like S fails. This is because the nature of the ioco-like relation checks only that the IUT may not emit any output symbol that was not enabled in the specification, after they both experience any sequence of events that runs on the specification. On the other hand, the definition of the ioco-like relation says nothing about input symbols that may be emitted by the IUT and the specification after a common run in both models.

Concluding Remarks
Testing conformance of reactive systems implementations is usually a hard task, due to the intrinsic nature of these systems, which allows for the asynchronous interactions of messages with an external environment. In such situations, the use of rigorous approaches capable of generating test suites for these models is indicated.
Previous studies focused on simple systems which have access only to a finite memory, represented by its states, e.g., LTS models. Here we studied a more powerful class of reactive systems, those that can make use of a potentially infinite memory, in the form of a pushdown stack. We extended the classical notion of ioco-conformance to cope with this new formalism that can make use of a potentially infinite memory. Essentially, this conformance relation still says that the implementation can only emit an output signal that is already present in the specification, after a common exchange of symbols has taken place in both models, but now both having access to a pushdown stack.
We developed, and proved correct, polynomial time algorithms that can be used to generate complete test suites and that can verify whether any implementation ioco-conforms to a given specification. In common practical situations the algorithms exhibit asymptotic worst case time complexity that can be bounded by O(n 3 + m 2 ) where n and m are proportional to the product of the number of states and transitions, respectively, present in the implementation and in the specification.
As discussed in Section 2 other works have also investigated recursive systems where a stack memory is present. They are, in general, based on classes of formal models which are proper subsets of Visibly Pushdown Languages, such as the class of Dyck languages. Therefore our approach is more complete and general in the sense that those systems and their respective models treated by our framework are also more powerful.
As additional work one could implement into a prototype the theoretical ideas developed here, and test the prototype with models that represent more practical situations. Also, one could investigate whether these ideas could be carried further using more powerful formal models, like some other forms of restricted PDAs, models where the communication channels have an infinite memory, or other formalisms that can represent timed events.