Software Testing in Safety Critical Systems

scheme Today, many sentry go-critical applications ar overcomeled by computer softw be package. at that placefore effective analyse as well asls ar mandatory to can a ut intimately degree of preventive and to reduce severe ills too minimum. The paper examines costing regulating hackneyeds in natural rubber-critical administrations. By comparing different softw be demonstrateing methods the requirements and ch wholeenges in safe-critical softwargon sampleing atomic frame 18 being evaluated. The QUICKIES measuring rod serves as the mall restrictive framework for all separately strategys and provides the basis for the creation of application- and Interdependently tankards.Moreover it defines certain asylum integrity levels depending on the study of application and recommends testing methods harmonize to these levels. In mock up- based rubber testing a consumption mystify with dependant space evince do of import is utilise to generate representative te st cases. Statistical testing is a mathematical approach that uses a high number of test cases to reach a signifi gaget result. The main challenge of all protective testing methods Is to reduce testing era and multiplexity without distorting the signifi give the gatece of the test.These can for example be transportation systems, power plants, and medical applications. As peoples lives depend on the correct function of such control systems and their softw be, thorough testing is required before they can be admitted to functioning. There ar many different software testing methods. Most of them all analyze the opportunity of a failure but do not value its severity. However, in condom-critical systems a failure that has severe consequences, even if it is exceedingly rare, can not be accepted. Therefore testing in this field has to be adopted tallyly.The purpose of this paper is to make up ones mind and par the in style(p) methods for sentry go-critical footwear testing an d to identify the most common industry standard in this field. Moreover the requirements and challenges in preventive-critical software testing allow be elaborated. At the beginning the paper will provide definitions that are required for the understanding of the subsequent chapters. After that, an introduction to the succus 508 recourse standard, which serves as a basis for most industry-specific standards, is given.The chapter examen modes will address several(prenominal) of the latest protective software testing methods in detail. 5 Definitions 2 Definitions 2. 1 Reliability and sentry duty In safety critical systems both, reliableness and safety are required to achieve the goals of dependability. However, reliability and safety are two different attributes of dependability. The reliability, R(t) , of a system is a function of time. It is be as the conditional prospect that the system will perform its mean function in a defined way over a given time period and under ce rtain specified and untrue conditions.The most employ parameter to characterize reliability is the Mean duration To Failure (MATT). The safety, S(t), of a system is defined as the probability that a system ill either perform its functions correctly or will discontinue its functions in a way that does not interrupt the operation of other systems or Jeopardize the safety of any people associated with the system 1. Based on these definitions, in reliability testing all failures are weighted equally, whereas in safety testing the failures are weighted match to their severity.Therefore, a reliable system may be quite dangerous and a safe system may be very unreliable. 2. 2 Safety-critical System supposes very complex to generate. As many states are unapproachable or very difficult to reach hey can be trim down to a relatively small number of representative system states. These states are grouped in three sub mystifys Normal State Subset (NUNS), Fail-Safe State subset (FPS) and Ris ky state subset (IRS). Their sexual congressships are s=Unusualness 6 Their inter-dependability is described as a Markova twine (see figure 1) 2. Figure 1 Three-state Markova Model for Safety-critical Systems(Source 2. Markova drawstring Usage Model The Markova chain usage feign describes the feasible usage of a software based on a predicted environment. It can be used to generate statistical test cases and to forecast the software reliability. In an Markova model the passageway from operation I to operation J can be denoted by an ordered pair . Let be the renewing probability from operation I to operation J, with and EX=I .. N p(is)=1, where n is the number of operations. The transitions and transition probabilities can be represented in the form of a matrix 3.Each specific usage of the program corresponds to a path X=(XI, XX, Xi) in the Markova chain where Xi corresponds to the I-the operation. P(Xi, X) determines the next penalize operation J after consummation of oper ation I. Since the operations are random rabbles, each path X=(XI, XX, ) forms a stochastic process. For a fibericular path x=(ox, XSL , ), the corresponding path execution probability is 3 7 pox pop , x 3 Standards There exist both national and international standards and guidelines at different depths and classifications which define requirements for safety-related technologies. Yester and provides the basis for the creation of application- and underspecified standards. It includes more than 500 pages of normative and informative specifications and proposals. present most safety-related standards are based on he JUICE 508 in combination with the previously applicable requirements 4. The JUICE 508 defines so called Safety Integrity Levels (Sills) which serve as a measure for the safety requirements on a certain system. The following table shows the different SILLS as intimately as the corresponding probability of failure and application examples.Probability of Failure unmatched Failure in x Years Consequences Application Example The sustain three split, are informative and include practical examples which should help to simplify the application of the standard. The CE 61 508 describes the complete life cycle of safety-related systems from planning to decommissioning and refers to all aspects related to the use and requirements for electrical / electronic / programmable electronic systems (E / E / PEE) for separately functions 4. According to the focus of this paper except the parts relating to software testing are mentioned in the following paragraph. Figure 2 shows the verification and validation process in software development according to the JUICE 508 standard.The E/E/PEE system safety requirements are applied both on the system architecture and the software specifications. all(prenominal) level in the system architecture verifies if it meets the requirements of the next higher seam (I. E. Coding fulfills module objective requirements, module des ign fulfills software yester design requirements etc. ). Moreover each system architecture layer is tried and true by a specific test. As soon as the test circuit is closed successfully the software can be validated. The standard also recommends and rates certain test methods according to the required SILL. In order to meet the requirements of the CE standard a series.Test methods comprised in the CE 61 508 are categorized as follows 6 Failure analysis (I. E. rationality consequence programs) Dynamic analysis and testing (I. E. Test case execution from model-based test case generation) Functional and black box testing (I. . par classes and foreplay partition testing, including boundary value analysis) Performance testing (I. E. retort timings and memory constraints) Static analysis (I. E. Static analysis of run time error behavior) 9 Figure 2 CE 61 508-3 stoppage and Validation Process(Source 10 Testing Methods 4 Testing Methods There are many different software testing method s.A detailed introduction to all different methods would be far beyond the scope of this paper. Therefore the indite will only mention two methods he deems most germane(predicate) in the field of safety-related software testing. Finally both methods are compared and their manageable application areas are evaluated. 4. 1 Model-based Safety Testing In model-based testing explicit behavior models that encode the intended behavior of a system and its environment are used. These models generate pairs of inserts and outputs. The output of such a model represents the expected output of the system under test (SOT). mineral model-based testing method. The system safety-related behavior is defined in the safety requirements specification. Test cases are derived from a safety model that is extracted from the SHUT and from stiff safety requirements. This model encodes the intended behavior and maps each possible input to the corresponding output. Safety test demandion criteria relate to t he functional safety of the safety- critical system, to the structure of the model (state coverage, transition coverage), and also to a tumesce defined set of system faults.Safety test case specifications are used to formalize the safety test selection criteria and render them operational. For the given safety model and the safety test case specification, an automatic safety test case generator and optimizer generates the safety test case suite. Finally, the concreted input part of a test case is submitted to the SHUT and the SOTs output is recorded. The concentration of the input part of a test case is performed by a safety test engine. Besides executing the safety case, it can also compare the output of the SHUT with the expected output as provided by the safety test case 6. 1 Figure 3 Model-based Safety Testing according Gang You et al. (Source Test Case Generation One of the most commonly tools for test case generation are model checking techniques. The main purpose of model ch ecking is to verify a formal safety holding (given as a logic formula) on a system model. In test case generation, model checking is used in order to find violations of certain formal safety properties. Safety models of safety-critical software systems may develop a huge number of states. Therefore the greatest challenge when employ a model match is to cope with the state space explosion.As a countermeasure, Gang You et al. s approach applies the safety model, which is derived from SHUT and certain safety requirements. The model 12 limits the number of states by splitting them into three subsets (NUNS, FPS, IRS) containing only representative states (see 2. X). Moreover the safety model encodes he intended behavior, and from its structure, safety test cases can be derived. It thereby restricts the possible inputs into the SHUT and the set of possible separately behaviors of the SOT.Hence, to reduce the amount of testing and guarantee the look of testing the model checker will s earch those most oft entered states and generate the corresponding safety test cases without searching the whole state spaces. The selection of states is based on the safety requirements (Sills). Generally speaking, the safety model can be seen as a test selection bar generate safety-related test cases. Figure 4 shows the corresponding go down chart. 1 . The system safety model in the form of a finite state machine (FSML) is transformed into the input language of the model checker tool (SPIN) 2.Each test requirement of a given safety criterion is formulated as a temporal logic look (LET). 3. Based on the Markova model of a system, the state space is shared out into three subsets. 4. In term of these subsets, the negation of each expression of the formula is confirm by the model checker. If there is an execution path in the model that does not satisfy the negated formula then it is presented by the model checker as a counter-example. This path becomes a test sequence that satis fies the master test requirement. 5.The inputs and outputs that form the executable test case are extracted from the counter-example or are derived by a corresponding guided pretension of the model. 13 Figure 4 Test Case Generation Framework according Gang You et al. (Source 4. 2 Statistical Testing As already mentioned in 2. 1 reliability is defined as the conditional probability that the system will perform its intended function. This chapter will link the reliability of a system with the Markova usage model (see 2. 3). Let f be a function that shows the failure probability of a software. The argument D represents the possible usage set of the software.Each element AXED is a usage path from quo (initial operation) to dismount (final operation) The relation between software reliability R and failure probability F is R=l -F (2). In the assumed model the failure behavior of the software only depends on its usage path X and not on the input. This means that the input domain corresp onding to the used X is homogeneous. The simplest way of obtaining unbiased reliability estimation of the software is to select N test paths XSL, XX, , CNN according to the usage model. The exult of the function f(Xi) is 1 if the path fails and O otherwise.Then the arithmetic 14 mean of f(Xi) is an unbiased estimate PEP(f(X)), which is the mathematical expectation of the software failure probability under transition matrix P. Hence, the software reliability can be expressed as R=l -PEP(f(X)) 3. Critical operations are infrequently penalize in real applications. This generates the problem that development organizations have to spend too much time when performing adequate statistical testing. Although one can overcome these drawbacks by increasing the execution probabilities of critical operations during statistical entire software under test. Yang Going et al. 3 found a possible approach to overcome this problem Importance Sampling (IS) Based Safety-critical software package Stati stical Testing speedup. IS Based Safety-critical Software Statistical Testing Acceleration This chapter presents the Is-based software statistical testing acceleration method. It ensures that the critical operations tested adequately by adjusting the transition probabilities in the matrix of the usage model, and at the same time, produces the unbiased reliability of the software under test. The IS technique reduces simulation run times hen estimating the probabilities of rare events by Monte Carlo simulations 3.For complex software with a large model matrix, the simulation procedure is practically extremely time consuming. To overcome this problem, Yang Going et al. s approach adopts a simulated annealing algorithm to calculate the optimal matrix Q. This wide used optimization method employs stochastic techniques to avoid being detain in local optimal solution. The 16 exact mathematical commentary of this algorithm is complex and would be out of the scope of this paper. 3 4. 3 Me thod Comparison Although model-based and statistical testing follow completely different approaches, the challenges are very similar.Both methods have to limit the extent and complexity of testing. Model-based testing reduces the number of test cases by restricting the state space domain of the Markova chain usage model. Whereas statistical testing reduces the number by changing the relation between critical and normal test cases with help off likeliness ratio. 5 Conclusion Today an increasing number of safety-critical applications are controlled by computer software. Therefore effective testing tools are required to provide a high degree of safety and to reduce severe failures to a minimum. The paper focused on