«

»

Pairwise Testing of Dynamic Composite Services

by Ajay Kattepur, Sagar Sen, Benoit Baudry, Albert Benveniste, Claude Jard
Abstract:
Online services encapsulate enterprises, people, software systems and often operate in poorly understood environments. Using such services in tandem to predictably orchestrate a complex task is one of the principal challenges of service-oriented computing. A composite service orchestration soliciting multiple atomic services is plagued by a number of sources of variation. For instance, availability of an atomic service and its response time are two important sources of variation. Moreover, the number of possible variations in a composite service increases exponentially with increase in the number of atomic services. Testing such a composite service presents a crucial challenge as its often very expensive to exhaustively examine the variation space. Can we effectively test the dynamic behavior of a composite service using only a subset of these variations? This is the question that intrigues us. In this paper, we first model composite service variability as a feature diagram (FD) that captures all valid configurations of its orchestration. Second, we apply pairwise testing to sample the set of all possible configurations to obtain a concise subset. Finally, we test the composite service for selected pairwise configurations for a variety of QoS metrics such as response time, data quality, and availability. Using two case studies, Car crash crisis management and eHealth management, we demonstrate that pairwise generation effectively samples the full range of QoS variations in a dynamic orchestration. The pairwise sampling technique eliminates over __MATH0__ redundancy in configurations, while still calling all atomic services at least once. We rigorously evaluate pairwise testing for the criteria such as: a) ability to sample the extreme QoS metrics of the service; b) stable behavior of the extracted configurations c) compact set of configurations that can help evaluate QoS tradeoffs and d) comparison with random sampling.
Reference:
Pairwise Testing of Dynamic Composite Services (Ajay Kattepur, Sagar Sen, Benoit Baudry, Albert Benveniste, Claude Jard), In Proceedings of the International Symposium on Software Engineering for Adaptive and Self Managing Systems (SEAMS), IEEE, 2011.
Bibtex Entry:
@inproceedings{Kattepur2011,
	Abstract = {  
    Online services encapsulate enterprises, people, software systems and often
    operate in poorly understood  environments. Using such services in tandem to
    predictably orchestrate a complex task is one of the principal challenges of
    service-oriented computing. A composite service orchestration soliciting
    multiple atomic services is plagued by a number of sources of variation. For
    instance, availability of an atomic service and its response time are two
    important sources of variation. Moreover, the number of possible variations in
    a composite service increases exponentially with increase in the number of
    atomic services. Testing such a composite service presents a crucial challenge
    as its often very expensive to exhaustively examine the variation space. Can
    we effectively test the dynamic behavior of a composite service using only a
    subset of these variations? This is the question that intrigues us. In this
    paper, we first model composite service variability as a feature diagram (FD)
    that captures all valid configurations of its orchestration.  Second, we apply
    pairwise testing to sample the set of all possible configurations to obtain a
    concise subset.  Finally, we test the composite service for selected pairwise
    configurations for a variety of QoS metrics such as response time, data
    quality, and availability. Using two case studies, Car crash crisis management
    and eHealth management, we demonstrate that pairwise generation   effectively
    samples the full range of QoS variations in a dynamic orchestration. The
    pairwise sampling technique  eliminates over $99%$ redundancy in
    configurations, while still calling all atomic services at least once. We
    rigorously evaluate pairwise testing for the criteria such as: a) ability to
    sample the extreme QoS metrics of the service; b) stable behavior of the
    extracted configurations c) compact set of configurations that can help
    evaluate QoS tradeoffs and d) comparison with random sampling.
      },
	Address = {Honolulu, Hawaii, USA},
	keywords = {test, adpative,SOA},
	Author = {Ajay Kattepur and Sagar Sen and Benoit Baudry and Albert Benveniste and Claude Jard},
	Booktitle = {Proceedings of the International Symposium on Software Engineering for Adaptive and Self Managing Systems (SEAMS)},
	Month = {May},
	Publisher = {IEEE},
	Title = {Pairwise Testing of Dynamic Composite Services},
	Url = {http://www.irisa.fr/triskell/perso_pro/bbaudry/publis/kattepur11.pdf},
	X-International-Audience = {yes},
	X-Language = {EN},
	X-Proceedings = {yes},
	Year = {2011},
	x-abbrv = {SEAMS},
	pages = {138 - 147},
	Bdsk-Url-1 = {http://www.irisa.fr/triskell/perso_pro/bbaudry/publis/kattepur11.pdf}}