|
|
Relativity in Software Engineering Measurements
Posted On August 25, 2007 by Rose Mary filed under Miscellaneous
Abstract: Though the relevance of the term relativity in software engineering measurements may appear strange as the term has been known to have been associated with the name of the famous physicist of the 20th century Albert Einstien for his landmark work on the theory of relativity. The basic principle behind the theory of relativity is that all the physical measurements are not absolute but relative. Similar phenomenon is observed in the measurements of various attributes of the software products, processes or projects. The value of LOC because of no standard definition of a line in source code differs from language to language. Similarly Halstead’s software Science metrics differs depending upon what implementation language is used. In this paper an investigation is made into such aspects of software engineering metrics and their overall impact on the software development activities.
1. Introduction:
You cannot control what you cannot measure. Measurement is a necessity for measuring and hence controlling various aspects of software and software related activities, like quality of the resulting software product under construction or the productivity of the project or improvement of software processes. All streams of engineering and physical sciences make use of measurements to quantify various characteristics of their products. Similarly a need of having measures to measure various aspects of software product, processes and projects have been realized to make it an exac science. Since software has no physical attributes, conventional measures are of no use and accordingly number of measures called metrics have been defined to quantify things like the size, complexity, reliability of a software product and other aspects relating to the activities in software development. Some of these characteristics can be measured directly and other aspects which cannot be measured directly are indirect measures. In case of a software product, metrics simply provide the scale for quantifying qualities. Actual measurement must be performed on a given software system in order to use metrics for quantifying characteristics of the given software.
2. Classification of Software Measuremens.
In software measurement activity there are three classes of entities of interest.
Processes: any software related activity which takes place over time.
Products: any artifact, deliverable or documents which arise out of the process.
Resources: Items which are inputs to processes.
There is a distinction between attributes of these, which are internal and external [6].
Internal Attributes: Internal attributes of a product, process or resource are those which can be measured purely in terms of the product, process or resource itself. For example length is an internal attribute of any software document, while elapsed time is an internal attribute of any software process.
External Attributes: external attributes of a product, process or resource are those which can only be measured with respect to how the product, process or resource relates to other entities in the environment. For example reliability of a program (a product itself) is dependent not just on the program itself, but on the compiler, machine and user. Productivity is an external attribute of a resource, namely people (either as individual or groups) it is clearly dependent on many aspects of the processes and the quality of products delivered.
Software managers and software users would like to measure and predict external attributes. Unfortunately they are necessarily only measurable indirectly. For example productivity of personnel is most commonly measured as a ratio of size of code delivered (an internal product attribute) and effort (an internal process attribute). The problem with this over simplistic measure of productivity has been well documented. Similarly “quality” of a software system (a very high level external product attribute) and size measured by KLOC [2], while reasonable for developers this measure of quality cannot be said to be a valid measure from the point of view of the user[6].
3. What is Measurement ?
Measurement is defined as the process by which number or symbols are assigned to attributes of entities in the real world in such away as to describe them, accordingly to clearly defined rules. [3], [4]. An activity may be an object such as a person or a software specification or an event such as a journey or the testing phase of a software project. An attribute is a feature or property of the entity such as the height or blood pressure of a person, the length or functionality (of a specification), the cost (of journey) or the duration (of testing phase).
In order that the measured values are unambiguous a model can be defined for the entities being measured. The model reflects a specific viewpoint. The need for good models are particularly relevant in software engineering measurement. For example even as simple a measure of length of programs as lines of code (LOC) requires well defined model of programs which enables us to identify unique lines unambiguously. Similarly the measure the effort spent on, say the unit listing process we would need an agreed “model” of the process which at least makes clear when the process begins and ends.
There are two broad types of measurements, direct and indirect. Direct measurement of an attribute is measurement which does not depend on the measurement of any other attribute. Indirect measurement of an attribute is measurement which involves the measurement of one or more other attributes. It turns out that while some attributes can be measured directly, we get more sophisticated measurement if we measure indirectly [6].
Uses of Measurements: Assessment and Prediction
There are two broad uses of measurements: for assessment and for prediction. Predictive measurement of an attribute A will generally depend on a mathematical model relating A to some existing measures of attributes
A1, A2….. An
Accurate predictive measurement is inevitably dependent upon careful measurement of attributesA1, A2. An. For example accurate estimate of the project resources are not obtained by simply applying a cost estimation model with fixed parameters. However careful measurement of key attributes of completed projects can lead to accurate resource predictions for future projects. Similarly it is possible to get accurate predictions of the reliability of software in operation but these are dependent on careful data collection relating to failure times during alpha-testing. [5]
For predictive measurement the model alone is not sufficient. Additionally we need to define the procedures for determining model parameters and interpreting the results.
Measurement activities must have clear objectives: the basic definition of measurement suggests that any measurement activity must proceed with very clear objectives or goals. First you need to know whether you want to measure for assessment or for prediction.
Next you need to know exactly which entities are the subject of interest then you need to decide which attributes of the chosen entities are the significant ones. The definition of measurement makes clear the need to specify both an entity and an attribute before any measurement can be undertaken [6].
4. Relativity in Software Measurements
Software measurement like measurement in any other discipline must adhere to the science of measurement if it is to gain widespread acceptance and validity. Another factor is the standardization of software measures causing no confusion or ambiguity like in LOC. Depending upon how you count the lines of source code you are likely to get a different value for LOC. Similar concerns have been raised by Churcher and Shepperd [9] and Martin Hitz and Behzad Montazeri [10] with the proposal of Chidamber and Kemerer’s Metrics Suite [11] for Object-Oriented Design. Similarly the use of syntax sensitive metrics like Halstead’s software science metric [7] will give different values for the same implementation depending upon the what implementation language is used.
Hallstead shows that length of the program can be approximated by the expression. N=n1 log2 n1+n2 log2 n2
And programme volume V may be defined as
V=N log2 n
n1 is the distinct operators.
n2 is the number of distinct operands.
n=n1+n2
It should be noted that V will vary with programming language and represents the volume of information (in bits) required to specify a program [8].
The phenomenon of relativity in software measurement occurs because of number of reasons. Where metrics can play a significant role in enhancing the quality of software products, for improving the productivity and accordingly tuning the software processes. At the same time, unless care is taken they can prove to be harmful too. Reasons other than language/platform differences causing relativity in software measurements are misinterpretation of data, improper/incomplete definition of metrics, lack of communication and training of the individuals employed for collecting metrics data and use of the commercial tools to collect metrics. In the following sections we show some of the situations where this can have a negative impact on the metrics programme.
4.1 Misinterpretation of Metrics Data.
Misinterpretation of data can lead one to draw wrong conclusions. For example if a programmer’s defect density increases despite quality improvement effort he/she might wrongly conclude that improvements are doing more harm than good and may revert to old ways of working.
4.2 Improper/Incomplete Metrics Definition.
Vague or ambiguous metric definitions cause different practitioners to interpret differently. For example time spent on fixing a bug found by testing is classified as test effort by one person, coding effort by another, and rework by a third. Same concerns have been raised by [9] in CK Metrics suite [11].
4.3 Lack of Communication and Training
Lack of communication and training can cause problems if the participants in the metrics program do not understand what exactly they are supposed to do. If people responsible for collecting metrics data do not understand the measurements and have not been trained in how to perform their tasks, they will not collect reliable data.
4.4 Tools Being Used to Collect Metrics Data
Most of the organizations make use of automated tools supplied by different vendors for collecting and interpreting data. Depending upon what tool you are using and how a specific metric has been implemented in that particular tool, the metrics values are likely to differ from tool to tool supplied by different vendors.
From the above observations it is clear that where implementation of a sensible metrics program can help manage software projects and organizations in an effective manner, care has to be taken to negate the negative impact caused by the relativity phenomenon in software measurements.
5. Conclusions
Induction of software metrics program in a software development organisation is a good step to increase the quality of software products and increase the productivity and improve software processes. There are hundreds of aspects of software products, projects and processes that can be measured to get very valuable information about the products, project and processes. But care has to be taken while deciding as to what should be measured and what should not be measured. Because of the inherent phenomenon of relativity in software measurement inferences are likely to differ from language to language or person to person and project to project. The phenomenon of relativity in software measurements can be harmful if proper care is not taking while implementing a metrics program. One should remember that an inverse relationship exits between quality and quantity unless care is taken. If the collected metrics such as LOC is used to measure the productivity of the programmer and accordingly take decisions for the reward or punishment it can be fatal as the person generating lesser LOC will be forced to change his/her behaviour thereby reducing the quality of the resulting code. Therefore while initiating a software metrics plan utmost attention has to be paid to evaluate the psychological impact it may have on the employees of the organisation. In software measurement it is a must that all the measurements have sufficient scientific basis and are free from the effect of relativity and ambiguity. For this we need to have a framework of standard measures to make a metric program in organisation successful.
References:
1. Fenton N. E. “Software Metrics: A Rigorous Approach:” London Chapman & Hall, 1991.
2. Inglis J. “Standard Software Quality Metrcs” AT&T Tech. Journal Vol. 65, no. 2 pp 113-118, 1985.
3. L. Finkelstein, “A Review of The Fundamental concepts measurement”, Measurement, Vol. 2 No. pp 25-34, 1984.
4. E. S. Roberts, “Measurement Theory with Applications to Decision Making, Utility and Social Science”, Reading M. A.: Addison Wesley, 1979.
5. S. Brocklehurst, P. V. Chan, B. Littlewood et. al.” Recaliberating Software Reliability Models” IEEE Transactions on Software Engineering. Vol. 16 No. 4 pp 458-470 Apr. 1990.
6. Fenton N. “Software Measurement: A Necessary Scientific Basis”, IEEE Transactions on Software Engineering Vol. No. 3 March 1994.
7. Halstead M., Elements of Software Science, North Holland 1977.
8. Pressman S. Roger, Software Engineering, A Practitioner’s Approach 5th edition, Mc GrawHill. 2001.
9. Churcher N. I. And M. J. Shepperd, “Comments on ‘A Metric Suite for Object-Oriented Design’, IEEE Transaction on Software Engineering, Vol. 21, no.3, pp 263-265, Mar, 1995.
10. Martin Hitz and Behzad Montazeri, Chidamber and Kemerer’s Metrics Suite: A Measurement Theory Perspective, IEEE Transactions on Software Engineering, Vol. 22, No. 4 April 1996.
11. S. R. Chidamber and C. F. Kemerer, “A Metrics Suite for Object-Oriented Design”, IEEE Transaction on Software Engineering, vol. 20, no.6, pp 476-493, June 1994
1. Introduction:
You cannot control what you cannot measure. Measurement is a necessity for measuring and hence controlling various aspects of software and software related activities, like quality of the resulting software product under construction or the productivity of the project or improvement of software processes. All streams of engineering and physical sciences make use of measurements to quantify various characteristics of their products. Similarly a need of having measures to measure various aspects of software product, processes and projects have been realized to make it an exac science. Since software has no physical attributes, conventional measures are of no use and accordingly number of measures called metrics have been defined to quantify things like the size, complexity, reliability of a software product and other aspects relating to the activities in software development. Some of these characteristics can be measured directly and other aspects which cannot be measured directly are indirect measures. In case of a software product, metrics simply provide the scale for quantifying qualities. Actual measurement must be performed on a given software system in order to use metrics for quantifying characteristics of the given software.
2. Classification of Software Measuremens.
In software measurement activity there are three classes of entities of interest.
Processes: any software related activity which takes place over time.
Products: any artifact, deliverable or documents which arise out of the process.
Resources: Items which are inputs to processes.
There is a distinction between attributes of these, which are internal and external [6].
Internal Attributes: Internal attributes of a product, process or resource are those which can be measured purely in terms of the product, process or resource itself. For example length is an internal attribute of any software document, while elapsed time is an internal attribute of any software process.
External Attributes: external attributes of a product, process or resource are those which can only be measured with respect to how the product, process or resource relates to other entities in the environment. For example reliability of a program (a product itself) is dependent not just on the program itself, but on the compiler, machine and user. Productivity is an external attribute of a resource, namely people (either as individual or groups) it is clearly dependent on many aspects of the processes and the quality of products delivered.
Software managers and software users would like to measure and predict external attributes. Unfortunately they are necessarily only measurable indirectly. For example productivity of personnel is most commonly measured as a ratio of size of code delivered (an internal product attribute) and effort (an internal process attribute). The problem with this over simplistic measure of productivity has been well documented. Similarly “quality” of a software system (a very high level external product attribute) and size measured by KLOC [2], while reasonable for developers this measure of quality cannot be said to be a valid measure from the point of view of the user[6].
3. What is Measurement ?
Measurement is defined as the process by which number or symbols are assigned to attributes of entities in the real world in such away as to describe them, accordingly to clearly defined rules. [3], [4]. An activity may be an object such as a person or a software specification or an event such as a journey or the testing phase of a software project. An attribute is a feature or property of the entity such as the height or blood pressure of a person, the length or functionality (of a specification), the cost (of journey) or the duration (of testing phase).
In order that the measured values are unambiguous a model can be defined for the entities being measured. The model reflects a specific viewpoint. The need for good models are particularly relevant in software engineering measurement. For example even as simple a measure of length of programs as lines of code (LOC) requires well defined model of programs which enables us to identify unique lines unambiguously. Similarly the measure the effort spent on, say the unit listing process we would need an agreed “model” of the process which at least makes clear when the process begins and ends.
There are two broad types of measurements, direct and indirect. Direct measurement of an attribute is measurement which does not depend on the measurement of any other attribute. Indirect measurement of an attribute is measurement which involves the measurement of one or more other attributes. It turns out that while some attributes can be measured directly, we get more sophisticated measurement if we measure indirectly [6].
Uses of Measurements: Assessment and Prediction
There are two broad uses of measurements: for assessment and for prediction. Predictive measurement of an attribute A will generally depend on a mathematical model relating A to some existing measures of attributes
A1, A2….. An
Accurate predictive measurement is inevitably dependent upon careful measurement of attributesA1, A2. An. For example accurate estimate of the project resources are not obtained by simply applying a cost estimation model with fixed parameters. However careful measurement of key attributes of completed projects can lead to accurate resource predictions for future projects. Similarly it is possible to get accurate predictions of the reliability of software in operation but these are dependent on careful data collection relating to failure times during alpha-testing. [5]
For predictive measurement the model alone is not sufficient. Additionally we need to define the procedures for determining model parameters and interpreting the results.
Measurement activities must have clear objectives: the basic definition of measurement suggests that any measurement activity must proceed with very clear objectives or goals. First you need to know whether you want to measure for assessment or for prediction.
Next you need to know exactly which entities are the subject of interest then you need to decide which attributes of the chosen entities are the significant ones. The definition of measurement makes clear the need to specify both an entity and an attribute before any measurement can be undertaken [6].
4. Relativity in Software Measurements
Software measurement like measurement in any other discipline must adhere to the science of measurement if it is to gain widespread acceptance and validity. Another factor is the standardization of software measures causing no confusion or ambiguity like in LOC. Depending upon how you count the lines of source code you are likely to get a different value for LOC. Similar concerns have been raised by Churcher and Shepperd [9] and Martin Hitz and Behzad Montazeri [10] with the proposal of Chidamber and Kemerer’s Metrics Suite [11] for Object-Oriented Design. Similarly the use of syntax sensitive metrics like Halstead’s software science metric [7] will give different values for the same implementation depending upon the what implementation language is used.
Hallstead shows that length of the program can be approximated by the expression. N=n1 log2 n1+n2 log2 n2
And programme volume V may be defined as
V=N log2 n
n1 is the distinct operators.
n2 is the number of distinct operands.
n=n1+n2
It should be noted that V will vary with programming language and represents the volume of information (in bits) required to specify a program [8].
The phenomenon of relativity in software measurement occurs because of number of reasons. Where metrics can play a significant role in enhancing the quality of software products, for improving the productivity and accordingly tuning the software processes. At the same time, unless care is taken they can prove to be harmful too. Reasons other than language/platform differences causing relativity in software measurements are misinterpretation of data, improper/incomplete definition of metrics, lack of communication and training of the individuals employed for collecting metrics data and use of the commercial tools to collect metrics. In the following sections we show some of the situations where this can have a negative impact on the metrics programme.
4.1 Misinterpretation of Metrics Data.
Misinterpretation of data can lead one to draw wrong conclusions. For example if a programmer’s defect density increases despite quality improvement effort he/she might wrongly conclude that improvements are doing more harm than good and may revert to old ways of working.
4.2 Improper/Incomplete Metrics Definition.
Vague or ambiguous metric definitions cause different practitioners to interpret differently. For example time spent on fixing a bug found by testing is classified as test effort by one person, coding effort by another, and rework by a third. Same concerns have been raised by [9] in CK Metrics suite [11].
4.3 Lack of Communication and Training
Lack of communication and training can cause problems if the participants in the metrics program do not understand what exactly they are supposed to do. If people responsible for collecting metrics data do not understand the measurements and have not been trained in how to perform their tasks, they will not collect reliable data.
4.4 Tools Being Used to Collect Metrics Data
Most of the organizations make use of automated tools supplied by different vendors for collecting and interpreting data. Depending upon what tool you are using and how a specific metric has been implemented in that particular tool, the metrics values are likely to differ from tool to tool supplied by different vendors.
From the above observations it is clear that where implementation of a sensible metrics program can help manage software projects and organizations in an effective manner, care has to be taken to negate the negative impact caused by the relativity phenomenon in software measurements.
5. Conclusions
Induction of software metrics program in a software development organisation is a good step to increase the quality of software products and increase the productivity and improve software processes. There are hundreds of aspects of software products, projects and processes that can be measured to get very valuable information about the products, project and processes. But care has to be taken while deciding as to what should be measured and what should not be measured. Because of the inherent phenomenon of relativity in software measurement inferences are likely to differ from language to language or person to person and project to project. The phenomenon of relativity in software measurements can be harmful if proper care is not taking while implementing a metrics program. One should remember that an inverse relationship exits between quality and quantity unless care is taken. If the collected metrics such as LOC is used to measure the productivity of the programmer and accordingly take decisions for the reward or punishment it can be fatal as the person generating lesser LOC will be forced to change his/her behaviour thereby reducing the quality of the resulting code. Therefore while initiating a software metrics plan utmost attention has to be paid to evaluate the psychological impact it may have on the employees of the organisation. In software measurement it is a must that all the measurements have sufficient scientific basis and are free from the effect of relativity and ambiguity. For this we need to have a framework of standard measures to make a metric program in organisation successful.
References:
1. Fenton N. E. “Software Metrics: A Rigorous Approach:” London Chapman & Hall, 1991.
2. Inglis J. “Standard Software Quality Metrcs” AT&T Tech. Journal Vol. 65, no. 2 pp 113-118, 1985.
3. L. Finkelstein, “A Review of The Fundamental concepts measurement”, Measurement, Vol. 2 No. pp 25-34, 1984.
4. E. S. Roberts, “Measurement Theory with Applications to Decision Making, Utility and Social Science”, Reading M. A.: Addison Wesley, 1979.
5. S. Brocklehurst, P. V. Chan, B. Littlewood et. al.” Recaliberating Software Reliability Models” IEEE Transactions on Software Engineering. Vol. 16 No. 4 pp 458-470 Apr. 1990.
6. Fenton N. “Software Measurement: A Necessary Scientific Basis”, IEEE Transactions on Software Engineering Vol. No. 3 March 1994.
7. Halstead M., Elements of Software Science, North Holland 1977.
8. Pressman S. Roger, Software Engineering, A Practitioner’s Approach 5th edition, Mc GrawHill. 2001.
9. Churcher N. I. And M. J. Shepperd, “Comments on ‘A Metric Suite for Object-Oriented Design’, IEEE Transaction on Software Engineering, Vol. 21, no.3, pp 263-265, Mar, 1995.
10. Martin Hitz and Behzad Montazeri, Chidamber and Kemerer’s Metrics Suite: A Measurement Theory Perspective, IEEE Transactions on Software Engineering, Vol. 22, No. 4 April 1996.
11. S. R. Chidamber and C. F. Kemerer, “A Metrics Suite for Object-Oriented Design”, IEEE Transaction on Software Engineering, vol. 20, no.6, pp 476-493, June 1994