| dc.contributor | D'AMORIM, Marcelo Bezerra | |
| dc.contributor | http://lattes.cnpq.br/3762670242328435 | |
| dc.creator | SOUTO, Sabrina de Figueirêdo | |
| dc.date | 2016-03-15T15:21:11Z | |
| dc.date | 2016-03-15T15:21:11Z | |
| dc.date | 2015-03-31 | |
| dc.identifier | https://repositorio.ufpe.br/handle/123456789/15975 | |
| dc.description | Software Product Lines (SPLs) allow engineers to systematically build families of
software products, defined by a unique combination of features—increments in functionality,
improving both the efficiency of the software development process and the quality of the software
developed. However, testing these kinds of systems is challenging, as it may require running each
test against a combinatorial number of products. We call this problem the High Dimensionality
Problem. Another obstacle to product line testing is the absence of Feature Models (FMs),
making it difficult to discover the real causes for test failures. We call this problem the Lack of
Feature Model Problem.
The High Dimensionality Problem is associated to the large space of possible configurations
that an SPL can reach. If an SPL has n boolean features, for example, there are 2n
possible feature combinations. Therefore, systematically testing this kind of system may require
running each test against all those combinations, in the worst case. The Lack of Feature Model
Problem is related to the absence of feature models. The FM enables accurate categorization of
failing tests as failures of programs or the tests themselves, not as failures due to inconsistent
combinations of features. For this reason, the lack of FM presents a huge challenge to discover
the true causes for test failures.
Aiming to solve these problems, we propose two lightweight techniques: SPLat and
SPLif. SPLat is a new approach to dynamically prune irrelevant configurations: the configurations
to run for a test can be determined during test execution by monitoring accesses to
configuration variables. As a result, SPLat reduces the number of configurations. Consequently,
SPLat is lightweight compared to prior works that used static analysis and heavyweight dynamic
execution. SPLif is a technique for testing SPLs that does not require a priori availability of
feature models. Our insight is to use a profile of passing and failing test runs to quickly identify
test failures that are indicative of a problem (in test or code) as opposed to a manifestation of
execution against an inconsistent combination of features.
Experimental results show that SPLat effectively identifies relevant configurations with
a low overhead. We also apply SPLat on two large configurable systems (Groupon and GCC),
and it scaled without much engineering effort. Experimental results demonstrate that SPLif
is useful and effective to quickly find tests that fail on consistent configurations, regardless of
how complete the FMs are. Furthermore, we evaluated SPLif on one large extensively tested
configurable system, GCC, where it helped to reveal 5 new bugs, 3 of which have been fixed
after our bug reports. | |
| dc.description | Software Product Lines (SPLs) allow engineers to systematically build families of
software products, defined by a unique combination of features—increments in functionality,
improving both the efficiency of the software development process and the quality of the software
developed. However, testing these kinds of systems is challenging, as it may require running each
test against a combinatorial number of products. We call this problem the High Dimensionality
Problem. Another obstacle to product line testing is the absence of Feature Models (FMs),
making it difficult to discover the real causes for test failures. We call this problem the Lack of
Feature Model Problem.
The High Dimensionality Problem is associated to the large space of possible configurations
that an SPL can reach. If an SPL has n boolean features, for example, there are 2n
possible feature combinations. Therefore, systematically testing this kind of system may require
running each test against all those combinations, in the worst case. The Lack of Feature Model
Problem is related to the absence of feature models. The FM enables accurate categorization of
failing tests as failures of programs or the tests themselves, not as failures due to inconsistent
combinations of features. For this reason, the lack of FM presents a huge challenge to discover
the true causes for test failures.
Aiming to solve these problems, we propose two lightweight techniques: SPLat and
SPLif. SPLat is a new approach to dynamically prune irrelevant configurations: the configurations
to run for a test can be determined during test execution by monitoring accesses to
configuration variables. As a result, SPLat reduces the number of configurations. Consequently,
SPLat is lightweight compared to prior works that used static analysis and heavyweight dynamic
execution. SPLif is a technique for testing SPLs that does not require a priori availability of
feature models. Our insight is to use a profile of passing and failing test runs to quickly identify
test failures that are indicative of a problem (in test or code) as opposed to a manifestation of
execution against an inconsistent combination of features.
Experimental results show that SPLat effectively identifies relevant configurations with
a low overhead. We also apply SPLat on two large configurable systems (Groupon and GCC),
and it scaled without much engineering effort. Experimental results demonstrate that SPLif
is useful and effective to quickly find tests that fail on consistent configurations, regardless of
how complete the FMs are. Furthermore, we evaluated SPLif on one large extensively tested
configurable system, GCC, where it helped to reveal 5 new bugs, 3 of which have been fixed
after our bug reports. | |
| dc.format | application/pdf | |
| dc.language | por | |
| dc.publisher | Universidade Federal de Pernambuco | |
| dc.publisher | UFPE | |
| dc.publisher | Brasil | |
| dc.publisher | Programa de Pos Graduacao em Ciencia da Computacao | |
| dc.rights | Attribution-NonCommercial-NoDerivs 3.0 Brazil | |
| dc.rights | http://creativecommons.org/licenses/by-nc-nd/3.0/br/ | |
| dc.subject | Linhas de Produtos de Software | |
| dc.subject | Sistemas Configuráveis | |
| dc.subject | Teste de Software e Depuração | |
| dc.subject | Feature Model | |
| dc.subject | Software Product Lines | |
| dc.subject | Configurable Systems | |
| dc.subject | Software Testing and Debugging | |
| dc.subject | Feature Model | |
| dc.title | Addressing high dimensionality and lack of feature models in testing of software product lines | |
| dc.type | doctoralThesis | |
