Time filter

Source Type

Lakhotia K.,King's College London | Harman M.,King's College London | Gross H.,Berner and Mattner Systemtechnik GmbH
Proceedings - 2nd International Symposium on Search Based Software Engineering, SSBSE 2010 | Year: 2010

Despite the large number of publications on Search-Based Software Testing (SBST), there remain few publicly available tools. This paper introduces AUSTIN, a publicly available SBST tool for the C language. The paper validates the tool with an empirical study of its effectiveness and efficiency in achieving branch coverage compared to random testing and the Evolutionary Testing Framework (ETF), which is used in-house by Daimler and others for Evolutionary Testing. The programs used in the study consist of eight non-trivial, real-world C functions drawn from three embedded automotive software modules. For the majority of the functions, AUSTIN is at least as effective (in terms of achieved branch coverage) as the ETF, and is considerably more efficient. © 2010 IEEE.


McMinn P.,University of Sheffield | Harman M.,University College London | Lakhotia K.,University College London | Hassoun Y.,King's College London | Wegener J.,Berner and Mattner Systemtechnik GmbH
IEEE Transactions on Software Engineering | Year: 2012

Search-Based Test Data Generation reformulates testing goals as fitness functions so that test input generation can be automated by some chosen search-based optimization algorithm. The optimization algorithm searches the space of potential inputs, seeking those that are fit for purpose, guided by the fitness function. The search space of potential inputs can be very large, even for very small systems under test. Its size is, of course, a key determining factor affecting the performance of any search-based approach. However, despite the large volume of work on Search-Based Software Testing, the literature contains little that concerns the performance impact of search space reduction. This paper proposes a static dependence analysis derived from program slicing that can be used to support search space reduction. The paper presents both a theoretical and empirical analysis of the application of this approach to open source and industrial production code. The results provide evidence to support the claim that input domain reduction has a significant effect on the performance of local, global, and hybrid search, while a purely random search is unaffected. © 1976-2012 IEEE.


Kruse P.M.,Berner and Mattner Systemtechnik GmbH | Schieferdecker I.,Fraunhofer Institute for Open Communication Systems
2012 Federated Conference on Computer Science and Information Systems, FedCSIS 2012 | Year: 2012

Due to limited test resources, it is often necessary to prioritize and select test cases for a given system under test. Although test case prioritization is well studied and understood, its combination with test data generation is difficult and not completely solved yet. For example, the Classification Tree Method is a well established method for test data generation, however the application of prioritization techniques to it is a current research topic. We present an extension of the classification tree method that allows the generation of optimized test suites, containing test cases ordered according to their importance with respect to test goals. The presented algorithms are incorporated into the Classification Tree Editor and empirically evaluated on a set of benchmarks. © 2012 Polish Info Processing Socit.


This article shows an overview of the new chapter "deployment" of EN 50128. A definition of deployment and its goals are given. Important aspects of deployment as well as its steps are discussed. Especially focus is set to the actors in the deployment phase.


Kruse P.M.,Berner and Mattner Systemtechnik GmbH | Wegener J.,Berner and Mattner Systemtechnik GmbH
Proceedings - IEEE 5th International Conference on Software Testing, Verification and Validation, ICST 2012 | Year: 2012

The combinatorial test design and combinatorial interaction testing are well studied topics. For the generation of dynamic test sequences from a formal specification of combinatorial problems, there has not been much work yet. The classification tree method implements aspects from the field of combinatorial testing. This paper extends the classification tree with additional information to allow the interpretation of the classification tree as a hierarchical concurrent state machine. Using this state machine, our new approach then uses a Multi-agent System to generate test sequences by finding and rating valid paths through the state machine. © 2012 IEEE.


Lindlar F.,TU Berlin | Windisch A.,TU Berlin | Wegener J.,Berner and Mattner Systemtechnik GmbH
ICSTW 2010 - 3rd International Conference on Software Testing, Verification, and Validation Workshops | Year: 2010

Evolutionary Functional Testing (EFT) is a relatively recent approach to automating the testing process. The research presented in this paper aims at increasing the acceptability of EFT in industrial settings. An approach suitable for efficiently and effectively testing complex continuous control systems is introduced. The main focus is on generating realistic test stimuli, enabling interactivity between test driver and test object, and facilitating the process of designing a suitable fitness function. This is accomplished by integrating EFT with modelbased testing methodologies resulting in an intuitive testing approach that enables even testers not familiar with searchbased testing to achieve good results with limited effort. A test environment optimized for deployment in the industrial domain is introduced. Features of the test environment include the capability of automatically generating realistic continuous test data sets, interacting with the system under test during test execution, and automatically executing and evaluating large numbers of tests. A thorough case study using an adaptive cruise control system from the automotive domain is performed to assess the approach. Results of this work indicate high usability, efficiency, and effectiveness of the proposed method for testing complex embedded systems. © 2010 IEEE.


Kruse P.M.,Berner and Mattner Systemtechnik GmbH
Proceedings - 2016 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016 | Year: 2016

Combinatorial test design is a well-established practice for obtaining tests. Most approaches focus on the actual test generation and test size minimization. For execution of test cases, expected results and executable test scripts are required. We investigate both, how to determine expected results for test cases, ideally in an automated fashion, and ways for generic test script generation to allow for execution of combinatorial test suites. This paper provides a survey of current approaches to test oracles and test script generation in combinatorial testing. Illustration and running example is provided in terms of the Classification Tree Method. © 2016 IEEE.


Robinson-Mallett C.L.,Berner and Mattner Systemtechnik GmbH
2012 2nd IEEE International Workshop on Model-Driven Requirements Engineering, MoDRE 2012 - Proceedings | Year: 2012

The time-to-market pressure and the introduction of new development standards into automotive industry have created interest in systematic approaches across the whole product lifecycle. Model-based design and testing methods and tools have been successfully introduced into development processes of car manufacturers and suppliers. Surprisingly, in practice the elicitation and specification of requirements remained largely unaffected by the introduction of model-based methods, while much effort has been spent on the introduction of functional specifications. As a consequence, where functional specifications exist, these are often created unsystematically and are of poor quality, leading to further problems during design, implementation and testing. In this paper we present an approach to improve specification quality through systemization of specification structures based on architectural block diagrams, behavioural statecharts and propositional logic structures. The approach has been implemented in a tool. Experiences from the application of approach and tool in industrial projects are reported. © 2012 IEEE.


Robinson-Mallett C.,Berner and Mattner Systemtechnik GmbH
ACM International Conference Proceeding Series | Year: 2014

Currently, the automotive industry is in a phase of reorientation and reorganization regarding security risks and hazards of in-car electronic systems. So far, security risks in the electronics development for automobiles were largely confined to configuration protection, e.g. preventing the manipulation of speedometers or disabling of power limitations. Now, the introduction of wireless information and communication technologies in vehicles leads to new challenges for the development and protection of the entire vehicle electronics. In this contribution a comparison of safety and security standards, on the example of ISO 26262 and ISO 15408, is presented and both standards are discussed regarding their industrial applicability and compatibility. A coordination scheme for security and safety engineering processes is proposed. Copyright is held by the owner/author(s).


Lakhotia K.,University College London | Harman M.,University College London | Gross H.,Berner and Mattner Systemtechnik GmbH
Information and Software Technology | Year: 2013

Context: Despite the large number of publications on Search-Based Software Testing (SBST), there remain few publicly available tools. This paper introduces AUSTIN, a publicly available open source SBST tool for the C language. 1 The paper is an extension of previous work [1]. It includes a new hill climb algorithm implemented in AUSTIN and an investigation into the effectiveness and efficiency of different pointer handling techniques implemented by AUSTIN's test data generation algorithms. Objective: To evaluate the different search algorithms implemented within AUSTIN on open source systems with respect to effectiveness and efficiency in achieving branch coverage. Further, to compare AUSTIN against a non-publicly available, state-of-the-art Evolutionary Testing Framework (ETF). Method: First, we use example functions from open source benchmarks as well as common data structure implementations to check if the decision procedure for pointer inputs, introduced in this paper, differs in terms of effectiveness and efficiency compared to a simpler alternative that generates random memory graphs. A second empirical study formulates two alternate hypotheses regarding the effectiveness and efficiency of AUSTIN compared to the ETF. These hypotheses are tested using a paired Wilcoxon test. Results and Conclusion: The first study highlights some practical problems with the decision procedure for pointer inputs described in this paper. In particular, if the code under test contains insufficient guard statements to enforce constraints over pointers, then using a constraint solver for pointer inputs may be suboptimal compared to a method that generates random memory graphs. The programs used in the second study do not require any constraint solving for pointer inputs and consist of eight non-trivial, real-world C functions drawn from three embedded automotive software modules. For these functions, AUSTIN is competitive compared to the ETF, achieving an equal or higher branch coverage for six of the functions. In addition, for functions where AUSTIN's branch coverage is equal or higher, AUSTIN is more efficient than the ETF. © 2012 Elsevier B.V. All rights reserved.

Loading Berner and Mattner Systemtechnik GmbH collaborators
Loading Berner and Mattner Systemtechnik GmbH collaborators