Empirical Software Engineering
Competency Leader
Empirical Software Engineering is a rapidly expanding sub-discipline within Software Engineering research. This is evidenced by the appearance and growth of dedicated forums for the publication of empirical research such as the ESEM (the international symposium on Empirical Software Engineering and Measurement [1]) and ESE (Empirical Software Engineering), a journal that has just recently increased its frequency to 6 issues per year and that has a citation index of 1.62 [2]. Indeed, while these are forums entirely dedicated to empirical software engineering research, publication of empirical observations and empirical evaluations is now widespread in prestigious software engineering and software maintenance forums like ICSE, ICSM and the IJSME [3],[4],[5],[6],[7],[8],[9],[10].
Empirical studies in software evolution can be characterized into 3 general areas, based on their rationale:
- Investigation of current techniques, tools and practice. These tend to act as assessments of current practice and suggest possible requirements for tools and techniques that will improve the current state of the art. Examples include [4], [9],[15],[16],[17];
- Investigation of trends over time. These studies act both in a descriptive and predictive capacity, where existing trends are extrapolated to identify the important issues arising in Software Engineering. Examples include [7], [11], [18], [19], [20],[21], [26];
- Evaluation of proposed software evolution approaches. Here proposed improvements to techniques, tools and practice are evaluated to assess their efficacy. Examples include [10],[11],[22],[23],[24],[25], [27], [28],[29].
A wide variety of data capturing techniques and data analysis techniques are used in such empirical studies. Data gathering techniques include interviewing [15],[16], surveys, questionnaires, think-aloud [10], data-mining, metrics [11], and gathering various other performance indicators [10]. Data analysis techniques include both quantitative and qualitative approaches such as statistical analysis [11],[13], content analysis [13],[16], [14] action research [12],[18] , grounded theory [17], and ethnography.
However, with such a plethora of techniques available, one of the major issues in software evolution research is the choice of appropriate data gathering techniques and data analysis techniques for a given rationale. In a software evolution context this is highlighted by Cornelissen et al. [14] who state that “comparison of existing approaches is hindered by the absence of common assessment frameworks and benchmarks” (while noting that the precision and recall criteria used in ‘feature location’ evaluations make that area an exception).
Refs:
[1] http://www.esem-conferences.org/ (last accessed 17th November 2010)
[2] http://www.springer.com/computer/swe/journal/10664 (last accessed 17th November, 2010)
[3] MunozF., Baudry B., Delamare R., and Le Traon Y.. (2009). Inquiring the usage of aspect-oriented programming: An empirical study. Proceedings of ICSM.
[4] ShihabE., Ming Jiang Z., Hassan A.. (2009). Studying the use of developer IRC meetings in open source projects. Proceedings of ICSM.
[5] StarkeJ., Luce C., Sillito J.. (2009). Searching and skimming: An exploratory study. Proceedings of ICSM.
[6] KhomhF., Guéhéneuc Y-G., Antoniol G.. (2009). Playing roles in design patterns: An empirical descriptive and analytic study. Proceedings of ICSM.
[7] Ferrari F., Burrows R. Lemos O.A.L., Garcia A., Figueiredo E., Cacho N., Lopes F., Temudo N.M., Silva L. and Soares S.. (2010). An Exploratory Study of Fault-Proneness in Evolving Aspect-Oriented Programs, Proceedings of ICSE.
[8] Liebig J., Apel S. and Lengauer C., Kaestner C., and Schulze M.. (2010) Analysis of the Variability in Forty Preprocessor-Based Software Product Lines. Proceedings of ICSE.
[9] Gorschek T., Tempero E., and Aristotle L.A.. (2010). A large-scale empirical study of practitioners' use of object-oriented concepts. Proceedings of ICSE.
[10] Buckley J., LeGear A., Exton C., Cadogan R., Johnston T., Looby B., and Koschke R.. (2008). Encapsulating targeted component abstractions using software Reflexion Modelling. International Journal of Software Maintenance and Evolution: Research and Practice.
[11] Rosik J., Le Gear A., Buckley J., Ali Babar M., and Connolly D. (2010). Assessing Architectural Drift in Commercial Software Development: A Case Study. Software Practice and Experience.
[12] English M., Buckley J., and Cahill T.. (2010). A replicated and refined empirical study of the use of friends in C++ software. International Journal of Software and Systems
[13] O’Brien M., Buckley J., Shaft T.. (2004). Expectation-based, Inference-based, and Bottom-up Software Comprehension. International Journal of Software Maintenance and Evolution: Research and Practice.
[14] Cornelissen B., Zaidman A., van Deursen A., Moonen L., and Koschke R.. (2009). A Systematic Survey of Program Comprehension through Dynamic Analysis. IEEE Transactions on Software Engineering.
[15] Conboy, K. and B. Fitzgerald, B., Method and developer characteristics for effective agile method tailoring: a study of expert opinion, ACM Transactions on Software Engineering Methodology, 20(1), 2010
[16] Casey, V. and I. Richardson,, Uncovering the Reality within Virtual Software Teams, International Conference on Global Software Engineering, ICGSE06., Florianopolis, Florianopolis, Brazil, 16-19 October 2006, IEEE Computer Society.
[17] Coleman, G. and O'Connor, R. (2008), Investigating Software Process in Practice: A Grounded Theory Perspective, Journal of Systems and Software, 81(5), pp 772-784
[18] Mc Caffery, F., J. Burton and I. Richardson, Improving Software Risk Management in a Medical Device Company, 31st International Conference on Software Engineering, ICSE 2009 (Software Engineering in Practice Track), Vancouver, Canada, 16-24th May, 2009.
[19] Rabiser, R., O'Leary, P. and Richardson, I. (2010), Key Activities for Product Derivation in Software Product Lines Journal of Systems and Software (accepted)
[20] Mc Caffery, F., M. Pikkarainen and I. Richardson, AHAA –Agile, Hybrid Assessment Method for Automotive, Safety Critical SMEs 30th International Conference on Software Engineering, ICSE 2008 (Automotive Track), Leipzig, Germany, 14-16th May, 2008, pp 551-560.
[21] Conboy, K. (2009), Agility from First Principles: Reconstructing the Concept of Agility in Information Systems Development,Information Systems Research, 20(3), pp 329-354.
[22] Mc Caffery, F., J. Burton and I. Richardson, Risk Management Capability Model (RMCM) for the Development of Medical Device Software, Software Quality Journal, Volume 18, Issue 1, 2010
[23] Richardson, I., V. Casey, F. McCaffery, J. Burton, S. Beecham, A Process Framework for Global Software Development Teams (under review)
[24] O’hEocha, C., K. Conboy, and X. Wang, So You Think You are Agile? International Conference in XP and Agile Processes in Systems Development (XP2010)
[25] Mattsson, A., Lundell, B., Lings, B. and Fitzgerald, B. (2009) ', Linking Model Driven Development and Software Architecture: A Case Study, IEEE Transactions on Software Engineering, 35(1), pp 83-93
[26] C. Gresse von Wangenheim, J.C Hauck, R. Zoucas, C.A Salviano, F. McCaffery, F. Shull - Creating Software Process Capability/Maturity Models, IEEE Software, July-Aug. 2010 , Volume : 27 , Issue:4 , pp. 92 - 94
[27] F. McCaffery, G. Coleman, “Lightweight SPI Assessments: What is the real Cost?”. Software Process Improvement and Practice Journal - Wiley Publishers. Volume 14 Issue 5, Pages 271 – 278, Special Issue: Part 1: Special Issue on SPI Experiences and Innovation for Global Software Development, 2009
[28] F. Mc Caffery, I. Richardson, P. Moller, “Automotive-Adept: A lightweight assessment method for the Automotive software industry”: Software Process Improvement and Practice Journal, 2008; 13: pp: 345-353
[29] F.Mc Caffery, P.Taylor, G. Coleman, “‘Adept’: A Software Process Improvement (SPI) Assessment Method for Small Software Companies”, IEEE Software Special Issue of “SE Challenges in Small Software Organizations”, Jan/Feb 2007. pp. 24-31.
Printer-friendly version