«

»

Pairwise Testing for Software Product Lines: A Comparison of Two Approaches

by Gilles Perrouin, Sebastian Oster, Sagar Sen, Jacques Klein, Benoit Baudry, Yves Le Traon
Abstract:
Software Product Lines (SPL) are difficult to validate due to combinatorics induced by variability across their features. This leads to combinatorial explosion of the number of derivable products. Exhaustive testing in such a large space of products is infeasible. One possible option is to test SPLs by generating test configurations that cover all possible t feature interactions (t-wise). It dramatically reduces the number of test products while ensuring reasonable SPL coverage. In this paper, we report our experience on applying t-wise techniques for SPL with two independent toolsets developed by the authors. One focuses on generality and split the generation problem according to strategies. The other emphasizes on providing efficient generation. To evaluate the respective merits of the approaches, measures such as the number of generated test specifications or the similarity between them are provided. By applying these measures we were able to derive useful insights for pairwise and t-wise testing of product lines.
Reference:
Pairwise Testing for Software Product Lines: A Comparison of Two Approaches (Gilles Perrouin, Sebastian Oster, Sagar Sen, Jacques Klein, Benoit Baudry, Yves Le Traon), In Software Quality Journal, Springer, volume 20, 2012.
Bibtex Entry:
@article{Perrouin2012,
	Abstract = {Software Product Lines (SPL) are difficult to validate due to combinatorics
    induced by variability across their features. This leads to combinatorial
    explosion of the number of derivable products. Exhaustive testing in
    such a large space of products is infeasible. One possible option is to test SPLs
    by generating test configurations that cover all possible t feature interactions
    (t-wise). It dramatically reduces the number of test products while ensuring
    reasonable SPL coverage. In this paper, we report our experience on applying
    t-wise techniques for SPL with two independent toolsets developed by the authors.
    One  focuses on generality and split the generation problem
    according to strategies. The other emphasizes on providing efficient generation.
    To evaluate the respective merits of the approaches, measures such as
    the number of generated test specifications or the similarity between them are provided. By applying these measures we were able to derive useful insights
    for pairwise and t-wise testing of product lines.},
	keywords = {SPL, test},
	Author = {Perrouin, Gilles and Oster, Sebastian and Sen, Sagar and Klein, Jacques and Baudry, Benoit and Le Traon, Yves},
	Journal = {Software Quality Journal},
	Pages = {605-643},
	Publisher = {Springer},
	Title = {Pairwise Testing for Software Product Lines: A Comparison of Two Approaches},
	Volume = {20},
	number = {3}, 
	X-International-Audience = {yes},
	X-Language = {EN},
	url = {http://hal.inria.fr/docs/00/80/58/56/PDF/ICST-SPLC-SQJ2010-Ext.pdf},
	x-abbrv = {SQJ},
	Year = {2012}}