Svoboda | Graniru | BBC Russia | Golosameriki | Facebook

Article

Pareto efficient multi-objective test case selection

Authors:

Mark HarmanAuthors Info & Claims

ISSTA '07: Proceedings of the 2007 international symposium on Software testing and analysis

Pages 140 - 150

https://doi.org/10.1145/1273463.1273483

Published: 09 July 2007 Publication History

Abstract

Previous work has treated test case selection as a single objective optimisation problem. This paper introduces the concept of Pareto efficiency to test case selection. The Pareto efficient approach takes multiple objectives such as code coverage, past fault-detection history and execution cost, and constructs a group of non-dominating, equivalently optimal test case subsets. The paper describes the potential bene?ts of Pareto efficient multi-objective test case selection, illustrating with empirical studies of two and three objective formulations.

References

[1]

S. Bates and S. Horwitz. Incremental program testing using program dependence graphs. In Conference Record of the Twentieth ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages pages 384--396, Charleston, South Carolina, Jan. 10-13, 1993. ACM Press.

Digital Library

[2]

D. Binkley. Reducing the cost of regression testing by semantics guided test case selection. In ICSM '95: Proceedings of the International Conference on Software Maintenance page 251, Washington, DC, USA, 1995. IEEE Computer Society.

Digital Library

[3]

C. A. Coello Coello, D. A. Van Veldhuizen, and G. B. Lamont. Evolutionary Algorithms for Solving Multi-Objective Problems KluwerAcademic Publishers, New York, May 2002.

Digital Library

[4]

Y. Collette and P. Siarry. Multiobjective Optimization: Principles and Case Studies Springer, 2004.

[5]

K. Deb. Multi-Objective Optimization Using Evolutionary Algorithms Wiley, Chichester, UK, 2001.

Digital Library

[6]

K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan. A Fast Elitist Non-Dominated Sorting Genetic Algorithm for Multi-Objective Optimization: NSGA-II. In Proceedings of the Parallel Problem Solving from Nature VI Conference pages 849--858, Paris, France, 2000. Springer. Lecture Notes in Computer Science No. 1917.

Digital Library

[7]

H. Do, S. G. Elbaum, and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering: An International Journal 10(4):405--435, 2005.

Digital Library

[8]

S. G. Elbaum, A. G. Malishevsky, and G. Rothermel. Prioritizing test cases for regression testing. In International Symposium on Software Testing and Analysis pages 102--112. ACM Press, 2000.

Digital Library

[9]

M. J. Harrold, R. Gupta, and M. L. Soffa. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2(3):270--285, 1993.

Digital Library

[10]

L. W. H. K. N. Leung. Insight into regressin testing. In Proceedings of the Conference on Software Maintenance October 1989.

[11]

M. Hutchins, H. Foster, T. Goradia, and T. Ostrand. Experiments on the effectiveness of data ?ow-and control-?ow-based test adequacy criteria. In Proceedings of the 16th International Conference on Software Engineering pages 191--200. IEEE Computer Society Press, May 1994.

Digital Library

[12]

Z. Li, M. Harman, and R. Hierons. Meta-heuristic search algorithms for regression test case prioritization. IEEE Transactions on Software Engineering To Appear.

Digital Library

[13]

A. G. Malishevsky, J. R. Ruthruff, G. Rothermel, and S. Elbaum. Cost-cognizant test case prioritization. Technical report, Department of Computer Science and Engineering, University of Nebraska-Lincoln, March 2006.

[14]

N. Nethercote and J. Seward. Valgrind: A program supervision framework. In Proceedings of the Third Workshop on Runtime Verification Colorado, USA, July 2003. Boulder.

[15]

G. Rothermel and M. J. Harrold. Analyzing regression test selection techniques. IEEE Transactions on Software Engineering 22(8):529--551, Aug. 1996.

Digital Library

[16]

G. Rothermel, M. J. Harrold, J. Ostrin, and C. Hong. An empirical study of the effects of minimization on the fault detection capabilities of test suites. In ICSM pages 34--43, 1998.

Digital Library

[17]

G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Test case prioritization: An empirical study. In Proceedings; IEEE International Conference on Software Maintenance pages 179--188, Los Alamitos, California, USA, 1999. IEEE Computer Society Press.

Digital Library

[18]

F. Szidarovsky, M. E. Gershon, and L. Dukstein. Techniques for multiobjective decision making in systems management Elsevier, New York, 1986.

[19]

P. Tonella, P. Avesani, and A. Susi. Using the case-based ranking methodology for test case prioritization. In ICSM '06: Proceedings of the 22nd IEEE International Conference on Software Maintenance pages 123--133, Washington, DC, USA, 2006. IEEE Computer Society.

Digital Library

[20]

K. R. Walcott, M. L. Soffa, G. M. Kapfhammer, and R. S. Roos. Time aware test suite prioritization. In ISSTA '06: Proceedings of the 2006 international symposium on Software testing and analysis pages 1--12, New York, NY, USA, 2006. ACM Press.

Digital Library

[21]

W. E. Wong, J. R. Horgan, S. London, and A. P. Mathur. Effect of test set minimization on fault detection effectiveness. Software - Practice and Experience 28(4):347--369, 1998.

Digital Library

[22]

S. S. Yau and Z. Kishimoto. A method for revalidating modi?ed programs in the maintenance phase. In Proceedings of 11th International Computer Software and Applications Conference (COMPSAC '87)pages pp. 272--277, October 1987.

Cited By

Laaber CYue TAli S(2024)Evaluating Search-Based Software Microbenchmark PrioritizationIEEE Transactions on Software Engineering10.1109/TSE.2024.3380836(1-16)Online publication date: 2024
https://doi.org/10.1109/TSE.2024.3380836
Jin XFeng YWang CLiu YHu YGao YXia KGuo L(2024)PIPELINEASCODE: A CI/CD Workflow Management System through Configuration Files at ByteDance2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00109(1011-1022)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00109
Garg KShekhar S(2024)Test case prioritization based on fault sensitivity analysis using ranked NSGA-2International Journal of Information Technology10.1007/s41870-024-01868-016:5(2875-2881)Online publication date: 2-May-2024
https://doi.org/10.1007/s41870-024-01868-0
Show More Cited By

Index Terms

Pareto efficient multi-objective test case selection
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Empirical evaluation of pareto efficient multi-objective regression test case prioritisation
ISSTA 2015: Proceedings of the 2015 International Symposium on Software Testing and Analysis

The aim of test case prioritisation is to determine an ordering of test cases that maximises the likelihood of early fault revelation. Previous prioritisation techniques have tended to be single objective, for which the additional greedy algorithm is ...
Runtime analysis of a multi-objective evolutionary algorithm for obtaining finite approximations of Pareto fronts

Previous theoretical analyses of evolutionary multi-objective optimization (EMO) mostly focus on obtaining @?-approximations of Pareto fronts. However, in practical applications, an appropriate value of @? is critical but sometimes, for a multi-...
An effective approach for regression test case selection using pareto based multi-objective harmony search
SBST '18: Proceedings of the 11th International Workshop on Search-Based Software Testing

Regression testing is a way of catching bugs in new builds and releases to avoid the product risks. Corrective, progressive, retest all and selective regression testing are strategies to perform regression testing. Retesting all existing test cases is ...

Comments

Information & Contributors

Information

Published In

ISSTA '07: Proceedings of the 2007 international symposium on Software testing and analysis

July 2007

258 pages

ISBN:9781595937346

DOI:10.1145/1273463

General Chair:
David S. Rosenblum
University College London, United Kingdom
,
Program Chair:
Sebastian Elbaum
University of Nebraska-Lincoln, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ISSTA07

Sponsor:

ISSTA07: International Symposium on Software Testing and Analysis

July 9 - 12, 2007

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '24

Sponsor:
sigsoft

33rd ACM SIGSOFT International Symposium on Software Testing and Analysis

September 16 - 20, 2024

Vienna , Austria

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

249
Total Citations
View Citations
2,020
Total Downloads

Downloads (Last 12 months)75
Downloads (Last 6 weeks)6

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Laaber CYue TAli S(2024)Evaluating Search-Based Software Microbenchmark PrioritizationIEEE Transactions on Software Engineering10.1109/TSE.2024.3380836(1-16)Online publication date: 2024
https://doi.org/10.1109/TSE.2024.3380836
Jin XFeng YWang CLiu YHu YGao YXia KGuo L(2024)PIPELINEASCODE: A CI/CD Workflow Management System through Configuration Files at ByteDance2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00109(1011-1022)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00109
Garg KShekhar S(2024)Test case prioritization based on fault sensitivity analysis using ranked NSGA-2International Journal of Information Technology10.1007/s41870-024-01868-016:5(2875-2881)Online publication date: 2-May-2024
https://doi.org/10.1007/s41870-024-01868-0
Garg KShekhar S(2024)Optimizing test case prioritization through ranked NSGA-2 for enhanced fault sensitivity analysisInnovations in Systems and Software Engineering10.1007/s11334-024-00561-620:3(307-328)Online publication date: 20-Apr-2024
https://doi.org/10.1007/s11334-024-00561-6
Zarrad ABahsoon RManimaran P(2024)Optimizing regression testing with AHP-TOPSIS metric system for effective technical debt evaluationAutomated Software Engineering10.1007/s10515-024-00458-531:2Online publication date: 8-Jul-2024
https://doi.org/10.1007/s10515-024-00458-5
Borge-Diez DOrtega-Cabezas PColmenar-Santos ABlanes-Peiró J(2024)Software Validation Techniques in the Automotive SectorDevelopment and Testing of Vehicle Software and its Influence on Sustainable Transport10.1007/978-3-031-47630-3_2(31-118)Online publication date: 11-Jan-2024
https://doi.org/10.1007/978-3-031-47630-3_2
Jamil MNour MAlotaibi SHussain MHussaini SNaseer A(2023)Software Product Line Maintenance Using Multi-Objective Optimization TechniquesApplied Sciences10.3390/app1315901013:15(9010)Online publication date: 6-Aug-2023
https://doi.org/10.3390/app13159010
Jin XServant F(2023)HybridCISave: A Combined Build and Test Selection Approach in Continuous IntegrationACM Transactions on Software Engineering and Methodology10.1145/357603832:4(1-39)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3576038
Arrieta AValle PAgirre JSagardui G(2023)Some Seeds Are Strong: Seeding Strategies for Search-based Test Case SelectionACM Transactions on Software Engineering and Methodology10.1145/353218232:1(1-47)Online publication date: 13-Feb-2023
https://dl.acm.org/doi/10.1145/3532182
Querejeta MJee ELiu LValle PArrieta ARezabal M(2023)Search-based Test Case Selection for PLC Systems using Functional Block Diagram Programs2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE59848.2023.00040(228-239)Online publication date: 9-Oct-2023
https://doi.org/10.1109/ISSRE59848.2023.00040
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents